Note: This is test shard 8 of 8.
[==========] Running 7 tests from 6 test suites.
[----------] Global test environment set-up.
[----------] 1 test from TableRangeRebalancingTest
[ RUN ] TableRangeRebalancingTest.Basic
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240502 13:59:39.937388 24841 test_util.cc:268] Using random seed: 61191770
I20240502 13:59:39.944031 24841 ts_itest-base.cc:112] Starting cluster with:
I20240502 13:59:39.944095 24841 ts_itest-base.cc:113] --------------
I20240502 13:59:39.944113 24841 ts_itest-base.cc:114] 3 tablet servers
I20240502 13:59:39.944135 24841 ts_itest-base.cc:115] 3 replicas per TS
I20240502 13:59:39.944150 24841 ts_itest-base.cc:116] --------------
2024-05-02T13:59:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T13:59:39Z Disabled control of system clock
I20240502 13:59:39.953596 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:41679
--webserver_interface=127.24.66.126
--webserver_port=0
--builtin_ntp_servers=127.24.66.84:40047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:41679
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false with env {}
W20240502 13:59:40.056790 24853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:40.056993 24853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:40.057045 24853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:40.060046 24853 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:40.060103 24853 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 13:59:40.060125 24853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:40.060144 24853 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 13:59:40.060179 24853 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 13:59:40.063688 24853 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:40047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:41679
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:41679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.24853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:40.064548 24853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:40.065582 24853 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:40.073170 24859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:40.073872 24858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:40.073904 24861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:40.074693 24853 server_base.cc:1034] running on GCE node
I20240502 13:59:40.075106 24853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:40.075898 24853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:40.077037 24853 hybrid_clock.cc:648] HybridClock initialized: now 1714658380077014 us; error 36 us; skew 500 ppm
I20240502 13:59:40.078852 24853 webserver.cc:460] Webserver started at http://127.24.66.126:39061/ using document root <none> and password file <none>
I20240502 13:59:40.079355 24853 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:40.079403 24853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:40.079561 24853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:40.084599 24853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "52b188801b5746188dcc5230629eaa3f"
format_stamp: "Formatted at 2024-05-02 13:59:40 on dist-test-slave-5npg"
I20240502 13:59:40.085067 24853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "52b188801b5746188dcc5230629eaa3f"
format_stamp: "Formatted at 2024-05-02 13:59:40 on dist-test-slave-5npg"
I20240502 13:59:40.088244 24853 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20240502 13:59:40.090427 24867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:40.091373 24853 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:40.091493 24853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
uuid: "52b188801b5746188dcc5230629eaa3f"
format_stamp: "Formatted at 2024-05-02 13:59:40 on dist-test-slave-5npg"
I20240502 13:59:40.091590 24853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:40.107048 24853 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:40.107707 24853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:40.107896 24853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:40.114750 24920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:41679 every 8 connection(s)
I20240502 13:59:40.114755 24853 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:41679
I20240502 13:59:40.115834 24853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 13:59:40.118395 24921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:40.118809 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 24853
I20240502 13:59:40.118939 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 13:59:40.123809 24921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52b188801b5746188dcc5230629eaa3f: Bootstrap starting.
I20240502 13:59:40.125839 24921 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52b188801b5746188dcc5230629eaa3f: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:40.126492 24921 log.cc:824] T 00000000000000000000000000000000 P 52b188801b5746188dcc5230629eaa3f: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:40.128021 24921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52b188801b5746188dcc5230629eaa3f: No bootstrap required, opened a new log
I20240502 13:59:40.130650 24921 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52b188801b5746188dcc5230629eaa3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b188801b5746188dcc5230629eaa3f" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 41679 } }
I20240502 13:59:40.130837 24921 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52b188801b5746188dcc5230629eaa3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:40.130918 24921 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52b188801b5746188dcc5230629eaa3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b188801b5746188dcc5230629eaa3f, State: Initialized, Role: FOLLOWER
I20240502 13:59:40.131399 24921 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52b188801b5746188dcc5230629eaa3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b188801b5746188dcc5230629eaa3f" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 41679 } }
I20240502 13:59:40.131553 24921 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52b188801b5746188dcc5230629eaa3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 13:59:40.131618 24921 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52b188801b5746188dcc5230629eaa3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 13:59:40.131727 24921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52b188801b5746188dcc5230629eaa3f [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:40.132619 24921 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52b188801b5746188dcc5230629eaa3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b188801b5746188dcc5230629eaa3f" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 41679 } }
I20240502 13:59:40.132961 24921 leader_election.cc:302] T 00000000000000000000000000000000 P 52b188801b5746188dcc5230629eaa3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52b188801b5746188dcc5230629eaa3f; no voters:
I20240502 13:59:40.133262 24921 leader_election.cc:288] T 00000000000000000000000000000000 P 52b188801b5746188dcc5230629eaa3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 13:59:40.133653 24926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52b188801b5746188dcc5230629eaa3f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:40.133864 24926 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52b188801b5746188dcc5230629eaa3f [term 1 LEADER]: Becoming Leader. State: Replica: 52b188801b5746188dcc5230629eaa3f, State: Running, Role: LEADER
I20240502 13:59:40.134145 24926 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52b188801b5746188dcc5230629eaa3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b188801b5746188dcc5230629eaa3f" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 41679 } }
I20240502 13:59:40.134420 24921 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52b188801b5746188dcc5230629eaa3f [sys.catalog]: configured and running, proceeding with master startup.
I20240502 13:59:40.135772 24927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52b188801b5746188dcc5230629eaa3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52b188801b5746188dcc5230629eaa3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b188801b5746188dcc5230629eaa3f" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 41679 } } }
I20240502 13:59:40.135774 24928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52b188801b5746188dcc5230629eaa3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52b188801b5746188dcc5230629eaa3f. Latest consensus state: current_term: 1 leader_uuid: "52b188801b5746188dcc5230629eaa3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b188801b5746188dcc5230629eaa3f" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 41679 } } }
I20240502 13:59:40.135918 24928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52b188801b5746188dcc5230629eaa3f [sys.catalog]: This master's current role is: LEADER
I20240502 13:59:40.135918 24927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52b188801b5746188dcc5230629eaa3f [sys.catalog]: This master's current role is: LEADER
I20240502 13:59:40.136375 24938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 13:59:40.139091 24938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 13:59:40.143816 24938 catalog_manager.cc:1347] Generated new cluster ID: 4c0456e343d549f0b72a8ed5469386a4
I20240502 13:59:40.143888 24938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 13:59:40.149547 24938 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 13:59:40.150687 24938 catalog_manager.cc:1504] Loading token signing keys...
I20240502 13:59:40.164110 24938 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 52b188801b5746188dcc5230629eaa3f: Generated new TSK 0
I20240502 13:59:40.164974 24938 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 13:59:40.172832 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:41679
--builtin_ntp_servers=127.24.66.84:40047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 13:59:40.279525 24945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:40.279750 24945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:40.279825 24945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:40.282886 24945 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:40.282965 24945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:40.283059 24945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20240502 13:59:40.286682 24945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:40047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:41679
--never_fsync=true
--heap_profile_path=/tmp/kudu.24945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:40.287623 24945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:40.288769 24945 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:40.296869 24950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:40.297468 24953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:40.298233 24951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:40.298969 24945 server_base.cc:1034] running on GCE node
I20240502 13:59:40.299321 24945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:40.299863 24945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:40.301024 24945 hybrid_clock.cc:648] HybridClock initialized: now 1714658380301002 us; error 40 us; skew 500 ppm
I20240502 13:59:40.302685 24945 webserver.cc:460] Webserver started at http://127.24.66.65:34945/ using document root <none> and password file <none>
I20240502 13:59:40.303205 24945 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:40.303259 24945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:40.303442 24945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:40.305081 24945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "54fbce87fe064ab5a46db4b4197a093d"
format_stamp: "Formatted at 2024-05-02 13:59:40 on dist-test-slave-5npg"
I20240502 13:59:40.305521 24945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "54fbce87fe064ab5a46db4b4197a093d"
format_stamp: "Formatted at 2024-05-02 13:59:40 on dist-test-slave-5npg"
I20240502 13:59:40.308688 24945 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 13:59:40.310671 24959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:40.311599 24945 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:40.311694 24945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "54fbce87fe064ab5a46db4b4197a093d"
format_stamp: "Formatted at 2024-05-02 13:59:40 on dist-test-slave-5npg"
I20240502 13:59:40.311793 24945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:40.327601 24945 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:40.328279 24945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:40.328460 24945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:40.328995 24945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:40.329886 24945 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:40.329932 24945 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:40.329979 24945 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:40.330008 24945 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:40.338687 24945 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:34575
I20240502 13:59:40.338721 25073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:34575 every 8 connection(s)
I20240502 13:59:40.339599 24945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 13:59:40.347831 25074 heartbeater.cc:347] Connected to a master server at 127.24.66.126:41679
I20240502 13:59:40.348045 25074 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:40.348425 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 24945
I20240502 13:59:40.348510 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 13:59:40.348613 25074 heartbeater.cc:511] Master 127.24.66.126:41679 requested a full tablet report, sending...
I20240502 13:59:40.350024 24883 ts_manager.cc:194] Registered new tserver with Master: 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65:34575)
I20240502 13:59:40.350594 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:41679
--builtin_ntp_servers=127.24.66.84:40047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 13:59:40.351670 24883 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.65:57959
W20240502 13:59:40.458495 25077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:40.458730 25077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:40.458792 25077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:40.461752 25077 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:40.461817 25077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:40.461884 25077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20240502 13:59:40.465430 25077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:40047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:41679
--never_fsync=true
--heap_profile_path=/tmp/kudu.25077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:40.466297 25077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:40.467381 25077 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:40.475989 25085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:40.476083 25083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:40.476859 25082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:40.477494 25077 server_base.cc:1034] running on GCE node
I20240502 13:59:40.477859 25077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:40.478371 25077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:40.479511 25077 hybrid_clock.cc:648] HybridClock initialized: now 1714658380479497 us; error 28 us; skew 500 ppm
I20240502 13:59:40.481379 25077 webserver.cc:460] Webserver started at http://127.24.66.66:39065/ using document root <none> and password file <none>
I20240502 13:59:40.481892 25077 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:40.481945 25077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:40.482097 25077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:40.483657 25077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1"
format_stamp: "Formatted at 2024-05-02 13:59:40 on dist-test-slave-5npg"
I20240502 13:59:40.484072 25077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1"
format_stamp: "Formatted at 2024-05-02 13:59:40 on dist-test-slave-5npg"
I20240502 13:59:40.487236 25077 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 13:59:40.489295 25091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:40.490231 25077 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:40.490325 25077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1"
format_stamp: "Formatted at 2024-05-02 13:59:40 on dist-test-slave-5npg"
I20240502 13:59:40.490422 25077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:40.513715 25077 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:40.514364 25077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:40.514515 25077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:40.515096 25077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:40.515952 25077 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:40.516016 25077 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:40.516085 25077 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:40.516119 25077 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:40.524322 25077 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:40803
I20240502 13:59:40.524375 25204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:40803 every 8 connection(s)
I20240502 13:59:40.525194 25077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 13:59:40.526680 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 25077
I20240502 13:59:40.526788 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 13:59:40.528856 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:41679
--builtin_ntp_servers=127.24.66.84:40047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 13:59:40.533373 25205 heartbeater.cc:347] Connected to a master server at 127.24.66.126:41679
I20240502 13:59:40.533576 25205 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:40.534031 25205 heartbeater.cc:511] Master 127.24.66.126:41679 requested a full tablet report, sending...
I20240502 13:59:40.534971 24883 ts_manager.cc:194] Registered new tserver with Master: 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66:40803)
I20240502 13:59:40.535593 24883 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.66:41207
W20240502 13:59:40.631027 25208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:40.631213 25208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:40.631283 25208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:40.634209 25208 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:40.634274 25208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:40.634357 25208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20240502 13:59:40.637944 25208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:40047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:41679
--never_fsync=true
--heap_profile_path=/tmp/kudu.25208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:40.638710 25208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:40.639683 25208 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:40.646919 25216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:40.647984 25208 server_base.cc:1034] running on GCE node
W20240502 13:59:40.648525 25213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:40.649240 25214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:40.649583 25208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:40.650087 25208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:40.651253 25208 hybrid_clock.cc:648] HybridClock initialized: now 1714658380651220 us; error 44 us; skew 500 ppm
I20240502 13:59:40.653033 25208 webserver.cc:460] Webserver started at http://127.24.66.67:39531/ using document root <none> and password file <none>
I20240502 13:59:40.653599 25208 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:40.653661 25208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:40.653829 25208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:40.655341 25208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "f09f152a68f54c518c0c8e931fed5941"
format_stamp: "Formatted at 2024-05-02 13:59:40 on dist-test-slave-5npg"
I20240502 13:59:40.655788 25208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "f09f152a68f54c518c0c8e931fed5941"
format_stamp: "Formatted at 2024-05-02 13:59:40 on dist-test-slave-5npg"
I20240502 13:59:40.658954 25208 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 13:59:40.660944 25222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:40.661839 25208 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:40.661939 25208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "f09f152a68f54c518c0c8e931fed5941"
format_stamp: "Formatted at 2024-05-02 13:59:40 on dist-test-slave-5npg"
I20240502 13:59:40.662019 25208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:40.680826 25208 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:40.681406 25208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:40.681555 25208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:40.682022 25208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:40.682832 25208 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:40.682879 25208 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:40.682936 25208 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:40.682963 25208 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:40.690855 25208 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:39311
I20240502 13:59:40.690901 25335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:39311 every 8 connection(s)
I20240502 13:59:40.691730 25208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 13:59:40.693878 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 25208
I20240502 13:59:40.693981 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TableRangeRebalancingTest.Basic.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 13:59:40.699734 25336 heartbeater.cc:347] Connected to a master server at 127.24.66.126:41679
I20240502 13:59:40.699903 25336 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:40.700335 25336 heartbeater.cc:511] Master 127.24.66.126:41679 requested a full tablet report, sending...
I20240502 13:59:40.701229 24883 ts_manager.cc:194] Registered new tserver with Master: f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:40.701742 24883 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.67:48299
I20240502 13:59:40.708043 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20240502 13:59:40.718421 24841 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20240502 13:59:41.354410 25074 heartbeater.cc:503] Master 127.24.66.126:41679 was elected leader, sending a full tablet report...
I20240502 13:59:41.537514 25205 heartbeater.cc:503] Master 127.24.66.126:41679 was elected leader, sending a full tablet report...
I20240502 13:59:41.703459 25336 heartbeater.cc:503] Master 127.24.66.126:41679 was elected leader, sending a full tablet report...
I20240502 13:59:44.471244 24883 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33952:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\200\007\001\000\000\000\000\000\006\001\000\000\000\000\000\007\001\000\377\377\377\177"
indirect_data: "<redacted>"""
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 8
seed: 0
}
}
W20240502 13:59:44.474311 24883 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240502 13:59:44.494465 25138 tablet_service.cc:1467] Processing CreateTablet for tablet b99e4b9853404fe188ec39b5d2a6c154 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 13:59:44.494688 25269 tablet_service.cc:1467] Processing CreateTablet for tablet b99e4b9853404fe188ec39b5d2a6c154 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 13:59:44.495347 25135 tablet_service.cc:1467] Processing CreateTablet for tablet 68513fdc8f5745088711a271afdbfe68 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 13:59:44.495182 25008 tablet_service.cc:1467] Processing CreateTablet for tablet 65bd1232d19e4a2c903fa19ede36a9ba (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 13:59:44.495788 25269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b99e4b9853404fe188ec39b5d2a6c154. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.495761 25134 tablet_service.cc:1467] Processing CreateTablet for tablet 0bd322784404440983b3ca1026f13c10 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 13:59:44.495963 25268 tablet_service.cc:1467] Processing CreateTablet for tablet 241895b448ae436b9b527a94c3331468 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 13:59:44.496148 25008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65bd1232d19e4a2c903fa19ede36a9ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.496218 25268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 241895b448ae436b9b527a94c3331468. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.496188 25133 tablet_service.cc:1467] Processing CreateTablet for tablet 2b5e884f76de4e749872a7b9a2bd8493 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 13:59:44.496579 25267 tablet_service.cc:1467] Processing CreateTablet for tablet adc3311b364245c69107c0cef3c5ce8a (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 13:59:44.496567 25132 tablet_service.cc:1467] Processing CreateTablet for tablet 1c75f6e566944a118ff217926b8b1f72 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 13:59:44.496793 25267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adc3311b364245c69107c0cef3c5ce8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.496979 25131 tablet_service.cc:1467] Processing CreateTablet for tablet cac3bbd142444b378b729de5cea6d1ae (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 13:59:44.497380 25130 tablet_service.cc:1467] Processing CreateTablet for tablet 8b30b6eada4345e8a8bc4e7bda88c8d5 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 13:59:44.497829 25129 tablet_service.cc:1467] Processing CreateTablet for tablet 2dbde6a2cb244667b7f24b6c4cb2c376 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 13:59:44.498261 25270 tablet_service.cc:1467] Processing CreateTablet for tablet 65bd1232d19e4a2c903fa19ede36a9ba (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 13:59:44.498499 25270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65bd1232d19e4a2c903fa19ede36a9ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.499575 25266 tablet_service.cc:1467] Processing CreateTablet for tablet 68513fdc8f5745088711a271afdbfe68 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 13:59:44.499788 25266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68513fdc8f5745088711a271afdbfe68. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.500597 25128 tablet_service.cc:1467] Processing CreateTablet for tablet d7b93d774c9d45858e14ed8fd6cc5bf2 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 13:59:44.500818 25265 tablet_service.cc:1467] Processing CreateTablet for tablet 0bd322784404440983b3ca1026f13c10 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 13:59:44.501030 25265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bd322784404440983b3ca1026f13c10. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.501055 25262 tablet_service.cc:1467] Processing CreateTablet for tablet cac3bbd142444b378b729de5cea6d1ae (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 13:59:44.501255 25262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cac3bbd142444b378b729de5cea6d1ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.502240 25007 tablet_service.cc:1467] Processing CreateTablet for tablet b99e4b9853404fe188ec39b5d2a6c154 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 13:59:44.502512 25007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b99e4b9853404fe188ec39b5d2a6c154. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.502859 25264 tablet_service.cc:1467] Processing CreateTablet for tablet 2b5e884f76de4e749872a7b9a2bd8493 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 13:59:44.503167 25263 tablet_service.cc:1467] Processing CreateTablet for tablet 1c75f6e566944a118ff217926b8b1f72 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 13:59:44.503479 25261 tablet_service.cc:1467] Processing CreateTablet for tablet 8b30b6eada4345e8a8bc4e7bda88c8d5 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 13:59:44.495072 25136 tablet_service.cc:1467] Processing CreateTablet for tablet adc3311b364245c69107c0cef3c5ce8a (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 13:59:44.494714 25137 tablet_service.cc:1467] Processing CreateTablet for tablet 241895b448ae436b9b527a94c3331468 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 13:59:44.503819 25260 tablet_service.cc:1467] Processing CreateTablet for tablet 2dbde6a2cb244667b7f24b6c4cb2c376 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 13:59:44.504141 25259 tablet_service.cc:1467] Processing CreateTablet for tablet d7b93d774c9d45858e14ed8fd6cc5bf2 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 13:59:44.504554 25263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c75f6e566944a118ff217926b8b1f72. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.504596 25006 tablet_service.cc:1467] Processing CreateTablet for tablet 241895b448ae436b9b527a94c3331468 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 13:59:44.494465 25139 tablet_service.cc:1467] Processing CreateTablet for tablet 65bd1232d19e4a2c903fa19ede36a9ba (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 13:59:44.504832 25006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 241895b448ae436b9b527a94c3331468. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.495785 25138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b99e4b9853404fe188ec39b5d2a6c154. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.505805 25135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68513fdc8f5745088711a271afdbfe68. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.505812 25261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b30b6eada4345e8a8bc4e7bda88c8d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.506124 25005 tablet_service.cc:1467] Processing CreateTablet for tablet adc3311b364245c69107c0cef3c5ce8a (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 13:59:44.506351 25005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adc3311b364245c69107c0cef3c5ce8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.506440 25137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 241895b448ae436b9b527a94c3331468. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.506799 25130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b30b6eada4345e8a8bc4e7bda88c8d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.507170 25129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dbde6a2cb244667b7f24b6c4cb2c376. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.507478 25260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dbde6a2cb244667b7f24b6c4cb2c376. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.507658 25004 tablet_service.cc:1467] Processing CreateTablet for tablet 68513fdc8f5745088711a271afdbfe68 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 13:59:44.507887 25004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68513fdc8f5745088711a271afdbfe68. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.507936 25132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c75f6e566944a118ff217926b8b1f72. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.509223 25003 tablet_service.cc:1467] Processing CreateTablet for tablet 0bd322784404440983b3ca1026f13c10 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 13:59:44.509471 25003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bd322784404440983b3ca1026f13c10. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.509945 25131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cac3bbd142444b378b729de5cea6d1ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.510056 25259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7b93d774c9d45858e14ed8fd6cc5bf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.511042 25002 tablet_service.cc:1467] Processing CreateTablet for tablet 2b5e884f76de4e749872a7b9a2bd8493 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 13:59:44.511277 25002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b5e884f76de4e749872a7b9a2bd8493. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.511847 25264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b5e884f76de4e749872a7b9a2bd8493. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.512284 25133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b5e884f76de4e749872a7b9a2bd8493. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.512527 25001 tablet_service.cc:1467] Processing CreateTablet for tablet 1c75f6e566944a118ff217926b8b1f72 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 3, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 13:59:44.512717 25001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c75f6e566944a118ff217926b8b1f72. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.513624 25000 tablet_service.cc:1467] Processing CreateTablet for tablet cac3bbd142444b378b729de5cea6d1ae (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 13:59:44.513801 25000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cac3bbd142444b378b729de5cea6d1ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.514151 25134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bd322784404440983b3ca1026f13c10. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.514780 24999 tablet_service.cc:1467] Processing CreateTablet for tablet 8b30b6eada4345e8a8bc4e7bda88c8d5 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 4, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 13:59:44.514954 24999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b30b6eada4345e8a8bc4e7bda88c8d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.516000 24997 tablet_service.cc:1467] Processing CreateTablet for tablet 2dbde6a2cb244667b7f24b6c4cb2c376 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 13:59:44.516108 25136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adc3311b364245c69107c0cef3c5ce8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.516181 24997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dbde6a2cb244667b7f24b6c4cb2c376. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.517210 24996 tablet_service.cc:1467] Processing CreateTablet for tablet d7b93d774c9d45858e14ed8fd6cc5bf2 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 5, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 13:59:44.517377 24996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7b93d774c9d45858e14ed8fd6cc5bf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.517585 25128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7b93d774c9d45858e14ed8fd6cc5bf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.519531 25356 tablet_bootstrap.cc:492] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d: Bootstrap starting.
I20240502 13:59:44.521063 25139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65bd1232d19e4a2c903fa19ede36a9ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.523010 25356 tablet_bootstrap.cc:654] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.523958 25356 log.cc:824] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:44.524818 25355 tablet_bootstrap.cc:492] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Bootstrap starting.
I20240502 13:59:44.526733 25269 tablet_service.cc:1467] Processing CreateTablet for tablet 34007af599ef4691977c361f01f22767 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 13:59:44.526988 25269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34007af599ef4691977c361f01f22767. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.527346 25259 tablet_service.cc:1467] Processing CreateTablet for tablet c8a7d300caa44b56918dc7045722e3c1 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 13:59:44.527346 24999 tablet_service.cc:1467] Processing CreateTablet for tablet c8a7d300caa44b56918dc7045722e3c1 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 13:59:44.527575 25259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8a7d300caa44b56918dc7045722e3c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.527575 24999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8a7d300caa44b56918dc7045722e3c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.528611 25139 tablet_service.cc:1467] Processing CreateTablet for tablet 34007af599ef4691977c361f01f22767 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 13:59:44.528599 25267 tablet_service.cc:1467] Processing CreateTablet for tablet 758640af315340bd8d9e919888d30c86 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 13:59:44.528604 24997 tablet_service.cc:1467] Processing CreateTablet for tablet 5617b9f3b6bb436e88a2aeaf051c4700 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 13:59:44.528827 25267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 758640af315340bd8d9e919888d30c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.528832 25139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34007af599ef4691977c361f01f22767. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.528844 24997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5617b9f3b6bb436e88a2aeaf051c4700. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.530754 25132 tablet_service.cc:1467] Processing CreateTablet for tablet 5617b9f3b6bb436e88a2aeaf051c4700 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 13:59:44.530797 25264 tablet_service.cc:1467] Processing CreateTablet for tablet 5617b9f3b6bb436e88a2aeaf051c4700 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 13:59:44.531000 25132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5617b9f3b6bb436e88a2aeaf051c4700. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.531000 25264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5617b9f3b6bb436e88a2aeaf051c4700. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.531384 25002 tablet_service.cc:1467] Processing CreateTablet for tablet 758640af315340bd8d9e919888d30c86 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 13:59:44.531625 25002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 758640af315340bd8d9e919888d30c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.532424 25354 tablet_bootstrap.cc:492] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941: Bootstrap starting.
I20240502 13:59:44.533490 25130 tablet_service.cc:1467] Processing CreateTablet for tablet c8a7d300caa44b56918dc7045722e3c1 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 7, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 13:59:44.533723 25130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8a7d300caa44b56918dc7045722e3c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.533852 25004 tablet_service.cc:1467] Processing CreateTablet for tablet 34007af599ef4691977c361f01f22767 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION -2147483648 <= VALUES < 0
I20240502 13:59:44.534067 25004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34007af599ef4691977c361f01f22767. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.534394 25355 tablet_bootstrap.cc:654] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.534803 25354 tablet_bootstrap.cc:654] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.535512 25355 log.cc:824] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:44.535934 25354 log.cc:824] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:44.536911 25135 tablet_service.cc:1467] Processing CreateTablet for tablet 758640af315340bd8d9e919888d30c86 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5ffa821e672546d3ab67b4dce63c0cfe]), partition=HASH (key) PARTITION 6, RANGE (key) PARTITION 0 <= VALUES < 2147483647
I20240502 13:59:44.537642 25356 tablet_bootstrap.cc:492] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d: No bootstrap required, opened a new log
I20240502 13:59:44.537842 25135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 758640af315340bd8d9e919888d30c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:44.537840 25356 ts_tablet_manager.cc:1367] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.002s
I20240502 13:59:44.537596 25355 tablet_bootstrap.cc:492] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 45c7ba84b0bb4626af59a8c46dcbb9c1: No bootstrap required, opened a new log
I20240502 13:59:44.538502 25355 ts_tablet_manager.cc:1367] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20240502 13:59:44.539054 25354 tablet_bootstrap.cc:492] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941: No bootstrap required, opened a new log
I20240502 13:59:44.539222 25354 ts_tablet_manager.cc:1367] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20240502 13:59:44.541198 25356 raft_consensus.cc:348] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.541582 25356 raft_consensus.cc:374] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.541697 25356 raft_consensus.cc:729] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54fbce87fe064ab5a46db4b4197a093d, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.542263 25356 consensus_queue.cc:260] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.542408 25354 raft_consensus.cc:348] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.542399 25355 raft_consensus.cc:348] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.542650 25355 raft_consensus.cc:374] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.542708 25354 raft_consensus.cc:374] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.542750 25355 raft_consensus.cc:729] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c7ba84b0bb4626af59a8c46dcbb9c1, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.542840 25354 raft_consensus.cc:729] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f09f152a68f54c518c0c8e931fed5941, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.543193 25355 consensus_queue.cc:260] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.543330 25356 ts_tablet_manager.cc:1398] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20240502 13:59:44.543295 25354 consensus_queue.cc:260] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.544009 25355 ts_tablet_manager.cc:1398] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240502 13:59:44.544185 25354 ts_tablet_manager.cc:1398] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20240502 13:59:44.544368 25355 tablet_bootstrap.cc:492] T cac3bbd142444b378b729de5cea6d1ae P 45c7ba84b0bb4626af59a8c46dcbb9c1: Bootstrap starting.
I20240502 13:59:44.545409 25355 tablet_bootstrap.cc:654] T cac3bbd142444b378b729de5cea6d1ae P 45c7ba84b0bb4626af59a8c46dcbb9c1: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.545665 25354 tablet_bootstrap.cc:492] T 65bd1232d19e4a2c903fa19ede36a9ba P f09f152a68f54c518c0c8e931fed5941: Bootstrap starting.
I20240502 13:59:44.546746 25354 tablet_bootstrap.cc:654] T 65bd1232d19e4a2c903fa19ede36a9ba P f09f152a68f54c518c0c8e931fed5941: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.547158 25356 tablet_bootstrap.cc:492] T 0bd322784404440983b3ca1026f13c10 P 54fbce87fe064ab5a46db4b4197a093d: Bootstrap starting.
I20240502 13:59:44.547238 25355 tablet_bootstrap.cc:492] T cac3bbd142444b378b729de5cea6d1ae P 45c7ba84b0bb4626af59a8c46dcbb9c1: No bootstrap required, opened a new log
I20240502 13:59:44.547331 25355 ts_tablet_manager.cc:1367] T cac3bbd142444b378b729de5cea6d1ae P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 13:59:44.547874 25355 raft_consensus.cc:348] T cac3bbd142444b378b729de5cea6d1ae P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.547940 25354 tablet_bootstrap.cc:492] T 65bd1232d19e4a2c903fa19ede36a9ba P f09f152a68f54c518c0c8e931fed5941: No bootstrap required, opened a new log
I20240502 13:59:44.547989 25355 raft_consensus.cc:374] T cac3bbd142444b378b729de5cea6d1ae P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.548025 25354 ts_tablet_manager.cc:1367] T 65bd1232d19e4a2c903fa19ede36a9ba P f09f152a68f54c518c0c8e931fed5941: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:44.548030 25355 raft_consensus.cc:729] T cac3bbd142444b378b729de5cea6d1ae P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c7ba84b0bb4626af59a8c46dcbb9c1, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.548148 25355 consensus_queue.cc:260] T cac3bbd142444b378b729de5cea6d1ae P 45c7ba84b0bb4626af59a8c46dcbb9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.548470 25355 ts_tablet_manager.cc:1398] T cac3bbd142444b378b729de5cea6d1ae P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:44.548480 25356 tablet_bootstrap.cc:654] T 0bd322784404440983b3ca1026f13c10 P 54fbce87fe064ab5a46db4b4197a093d: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.548653 25355 tablet_bootstrap.cc:492] T 2b5e884f76de4e749872a7b9a2bd8493 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Bootstrap starting.
I20240502 13:59:44.548851 25354 raft_consensus.cc:348] T 65bd1232d19e4a2c903fa19ede36a9ba P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.549005 25354 raft_consensus.cc:374] T 65bd1232d19e4a2c903fa19ede36a9ba P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.549098 25354 raft_consensus.cc:729] T 65bd1232d19e4a2c903fa19ede36a9ba P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f09f152a68f54c518c0c8e931fed5941, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.549299 25354 consensus_queue.cc:260] T 65bd1232d19e4a2c903fa19ede36a9ba P f09f152a68f54c518c0c8e931fed5941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.549590 25355 tablet_bootstrap.cc:654] T 2b5e884f76de4e749872a7b9a2bd8493 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.549775 25354 ts_tablet_manager.cc:1398] T 65bd1232d19e4a2c903fa19ede36a9ba P f09f152a68f54c518c0c8e931fed5941: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:44.549994 25354 tablet_bootstrap.cc:492] T 68513fdc8f5745088711a271afdbfe68 P f09f152a68f54c518c0c8e931fed5941: Bootstrap starting.
I20240502 13:59:44.550643 25356 tablet_bootstrap.cc:492] T 0bd322784404440983b3ca1026f13c10 P 54fbce87fe064ab5a46db4b4197a093d: No bootstrap required, opened a new log
I20240502 13:59:44.550740 25356 ts_tablet_manager.cc:1367] T 0bd322784404440983b3ca1026f13c10 P 54fbce87fe064ab5a46db4b4197a093d: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 13:59:44.551000 25354 tablet_bootstrap.cc:654] T 68513fdc8f5745088711a271afdbfe68 P f09f152a68f54c518c0c8e931fed5941: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.551342 25356 raft_consensus.cc:348] T 0bd322784404440983b3ca1026f13c10 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.551442 25356 raft_consensus.cc:374] T 0bd322784404440983b3ca1026f13c10 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.551477 25355 tablet_bootstrap.cc:492] T 2b5e884f76de4e749872a7b9a2bd8493 P 45c7ba84b0bb4626af59a8c46dcbb9c1: No bootstrap required, opened a new log
I20240502 13:59:44.551481 25356 raft_consensus.cc:729] T 0bd322784404440983b3ca1026f13c10 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54fbce87fe064ab5a46db4b4197a093d, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.551577 25355 ts_tablet_manager.cc:1367] T 2b5e884f76de4e749872a7b9a2bd8493 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 13:59:44.551612 25356 consensus_queue.cc:260] T 0bd322784404440983b3ca1026f13c10 P 54fbce87fe064ab5a46db4b4197a093d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.551898 25356 ts_tablet_manager.cc:1398] T 0bd322784404440983b3ca1026f13c10 P 54fbce87fe064ab5a46db4b4197a093d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 13:59:44.552064 25356 tablet_bootstrap.cc:492] T 241895b448ae436b9b527a94c3331468 P 54fbce87fe064ab5a46db4b4197a093d: Bootstrap starting.
I20240502 13:59:44.552270 25355 raft_consensus.cc:348] T 2b5e884f76de4e749872a7b9a2bd8493 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.552392 25355 raft_consensus.cc:374] T 2b5e884f76de4e749872a7b9a2bd8493 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.552436 25355 raft_consensus.cc:729] T 2b5e884f76de4e749872a7b9a2bd8493 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c7ba84b0bb4626af59a8c46dcbb9c1, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.552588 25355 consensus_queue.cc:260] T 2b5e884f76de4e749872a7b9a2bd8493 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.552899 25355 ts_tablet_manager.cc:1398] T 2b5e884f76de4e749872a7b9a2bd8493 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 13:59:44.553063 25355 tablet_bootstrap.cc:492] T 0bd322784404440983b3ca1026f13c10 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Bootstrap starting.
I20240502 13:59:44.553144 25356 tablet_bootstrap.cc:654] T 241895b448ae436b9b527a94c3331468 P 54fbce87fe064ab5a46db4b4197a093d: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.553402 25354 tablet_bootstrap.cc:492] T 68513fdc8f5745088711a271afdbfe68 P f09f152a68f54c518c0c8e931fed5941: No bootstrap required, opened a new log
I20240502 13:59:44.553498 25354 ts_tablet_manager.cc:1367] T 68513fdc8f5745088711a271afdbfe68 P f09f152a68f54c518c0c8e931fed5941: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 13:59:44.554008 25355 tablet_bootstrap.cc:654] T 0bd322784404440983b3ca1026f13c10 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.554098 25354 raft_consensus.cc:348] T 68513fdc8f5745088711a271afdbfe68 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.554204 25354 raft_consensus.cc:374] T 68513fdc8f5745088711a271afdbfe68 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.554236 25354 raft_consensus.cc:729] T 68513fdc8f5745088711a271afdbfe68 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f09f152a68f54c518c0c8e931fed5941, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.554348 25354 consensus_queue.cc:260] T 68513fdc8f5745088711a271afdbfe68 P f09f152a68f54c518c0c8e931fed5941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.554538 25356 tablet_bootstrap.cc:492] T 241895b448ae436b9b527a94c3331468 P 54fbce87fe064ab5a46db4b4197a093d: No bootstrap required, opened a new log
I20240502 13:59:44.554612 25356 ts_tablet_manager.cc:1367] T 241895b448ae436b9b527a94c3331468 P 54fbce87fe064ab5a46db4b4197a093d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 13:59:44.554658 25354 ts_tablet_manager.cc:1398] T 68513fdc8f5745088711a271afdbfe68 P f09f152a68f54c518c0c8e931fed5941: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:44.554831 25354 tablet_bootstrap.cc:492] T 1c75f6e566944a118ff217926b8b1f72 P f09f152a68f54c518c0c8e931fed5941: Bootstrap starting.
I20240502 13:59:44.555243 25356 raft_consensus.cc:348] T 241895b448ae436b9b527a94c3331468 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.555367 25356 raft_consensus.cc:374] T 241895b448ae436b9b527a94c3331468 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.555397 25356 raft_consensus.cc:729] T 241895b448ae436b9b527a94c3331468 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54fbce87fe064ab5a46db4b4197a093d, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.555506 25356 consensus_queue.cc:260] T 241895b448ae436b9b527a94c3331468 P 54fbce87fe064ab5a46db4b4197a093d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.555742 25354 tablet_bootstrap.cc:654] T 1c75f6e566944a118ff217926b8b1f72 P f09f152a68f54c518c0c8e931fed5941: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.555800 25356 ts_tablet_manager.cc:1398] T 241895b448ae436b9b527a94c3331468 P 54fbce87fe064ab5a46db4b4197a093d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 13:59:44.555974 25356 tablet_bootstrap.cc:492] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 54fbce87fe064ab5a46db4b4197a093d: Bootstrap starting.
I20240502 13:59:44.556600 25355 tablet_bootstrap.cc:492] T 0bd322784404440983b3ca1026f13c10 P 45c7ba84b0bb4626af59a8c46dcbb9c1: No bootstrap required, opened a new log
I20240502 13:59:44.556674 25355 ts_tablet_manager.cc:1367] T 0bd322784404440983b3ca1026f13c10 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 13:59:44.556813 25354 tablet_bootstrap.cc:492] T 1c75f6e566944a118ff217926b8b1f72 P f09f152a68f54c518c0c8e931fed5941: No bootstrap required, opened a new log
I20240502 13:59:44.556891 25354 ts_tablet_manager.cc:1367] T 1c75f6e566944a118ff217926b8b1f72 P f09f152a68f54c518c0c8e931fed5941: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:44.556924 25356 tablet_bootstrap.cc:654] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 54fbce87fe064ab5a46db4b4197a093d: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.557093 25355 raft_consensus.cc:348] T 0bd322784404440983b3ca1026f13c10 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.557196 25355 raft_consensus.cc:374] T 0bd322784404440983b3ca1026f13c10 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.557232 25355 raft_consensus.cc:729] T 0bd322784404440983b3ca1026f13c10 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c7ba84b0bb4626af59a8c46dcbb9c1, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.557344 25355 consensus_queue.cc:260] T 0bd322784404440983b3ca1026f13c10 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.557344 25354 raft_consensus.cc:348] T 1c75f6e566944a118ff217926b8b1f72 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.557479 25354 raft_consensus.cc:374] T 1c75f6e566944a118ff217926b8b1f72 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.557514 25354 raft_consensus.cc:729] T 1c75f6e566944a118ff217926b8b1f72 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f09f152a68f54c518c0c8e931fed5941, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.557622 25355 ts_tablet_manager.cc:1398] T 0bd322784404440983b3ca1026f13c10 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:44.557641 25354 consensus_queue.cc:260] T 1c75f6e566944a118ff217926b8b1f72 P f09f152a68f54c518c0c8e931fed5941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.557786 25355 tablet_bootstrap.cc:492] T adc3311b364245c69107c0cef3c5ce8a P 45c7ba84b0bb4626af59a8c46dcbb9c1: Bootstrap starting.
I20240502 13:59:44.557893 25354 ts_tablet_manager.cc:1398] T 1c75f6e566944a118ff217926b8b1f72 P f09f152a68f54c518c0c8e931fed5941: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:44.558039 25354 tablet_bootstrap.cc:492] T 241895b448ae436b9b527a94c3331468 P f09f152a68f54c518c0c8e931fed5941: Bootstrap starting.
I20240502 13:59:44.558727 25356 tablet_bootstrap.cc:492] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 54fbce87fe064ab5a46db4b4197a093d: No bootstrap required, opened a new log
I20240502 13:59:44.558746 25355 tablet_bootstrap.cc:654] T adc3311b364245c69107c0cef3c5ce8a P 45c7ba84b0bb4626af59a8c46dcbb9c1: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.558810 25356 ts_tablet_manager.cc:1367] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 54fbce87fe064ab5a46db4b4197a093d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 13:59:44.558962 25354 tablet_bootstrap.cc:654] T 241895b448ae436b9b527a94c3331468 P f09f152a68f54c518c0c8e931fed5941: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.559273 25356 raft_consensus.cc:348] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.559382 25356 raft_consensus.cc:374] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.559424 25356 raft_consensus.cc:729] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54fbce87fe064ab5a46db4b4197a093d, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.559551 25356 consensus_queue.cc:260] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 54fbce87fe064ab5a46db4b4197a093d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.559841 25356 ts_tablet_manager.cc:1398] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 54fbce87fe064ab5a46db4b4197a093d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 13:59:44.559990 25356 tablet_bootstrap.cc:492] T cac3bbd142444b378b729de5cea6d1ae P 54fbce87fe064ab5a46db4b4197a093d: Bootstrap starting.
I20240502 13:59:44.560009 25355 tablet_bootstrap.cc:492] T adc3311b364245c69107c0cef3c5ce8a P 45c7ba84b0bb4626af59a8c46dcbb9c1: No bootstrap required, opened a new log
I20240502 13:59:44.560076 25355 ts_tablet_manager.cc:1367] T adc3311b364245c69107c0cef3c5ce8a P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:44.560514 25355 raft_consensus.cc:348] T adc3311b364245c69107c0cef3c5ce8a P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.560611 25355 raft_consensus.cc:374] T adc3311b364245c69107c0cef3c5ce8a P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.560647 25355 raft_consensus.cc:729] T adc3311b364245c69107c0cef3c5ce8a P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c7ba84b0bb4626af59a8c46dcbb9c1, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.560778 25355 consensus_queue.cc:260] T adc3311b364245c69107c0cef3c5ce8a P 45c7ba84b0bb4626af59a8c46dcbb9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.561021 25355 ts_tablet_manager.cc:1398] T adc3311b364245c69107c0cef3c5ce8a P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:44.561165 25355 tablet_bootstrap.cc:492] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Bootstrap starting.
I20240502 13:59:44.561255 25356 tablet_bootstrap.cc:654] T cac3bbd142444b378b729de5cea6d1ae P 54fbce87fe064ab5a46db4b4197a093d: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.562104 25355 tablet_bootstrap.cc:654] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.562482 25354 tablet_bootstrap.cc:492] T 241895b448ae436b9b527a94c3331468 P f09f152a68f54c518c0c8e931fed5941: No bootstrap required, opened a new log
I20240502 13:59:44.562582 25354 ts_tablet_manager.cc:1367] T 241895b448ae436b9b527a94c3331468 P f09f152a68f54c518c0c8e931fed5941: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 13:59:44.563082 25354 raft_consensus.cc:348] T 241895b448ae436b9b527a94c3331468 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.563171 25354 raft_consensus.cc:374] T 241895b448ae436b9b527a94c3331468 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.563202 25354 raft_consensus.cc:729] T 241895b448ae436b9b527a94c3331468 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f09f152a68f54c518c0c8e931fed5941, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.563269 25356 tablet_bootstrap.cc:492] T cac3bbd142444b378b729de5cea6d1ae P 54fbce87fe064ab5a46db4b4197a093d: No bootstrap required, opened a new log
I20240502 13:59:44.563318 25355 tablet_bootstrap.cc:492] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 45c7ba84b0bb4626af59a8c46dcbb9c1: No bootstrap required, opened a new log
I20240502 13:59:44.563357 25356 ts_tablet_manager.cc:1367] T cac3bbd142444b378b729de5cea6d1ae P 54fbce87fe064ab5a46db4b4197a093d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 13:59:44.563378 25355 ts_tablet_manager.cc:1367] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:44.563329 25354 consensus_queue.cc:260] T 241895b448ae436b9b527a94c3331468 P f09f152a68f54c518c0c8e931fed5941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.563565 25354 ts_tablet_manager.cc:1398] T 241895b448ae436b9b527a94c3331468 P f09f152a68f54c518c0c8e931fed5941: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:44.563737 25354 tablet_bootstrap.cc:492] T 2dbde6a2cb244667b7f24b6c4cb2c376 P f09f152a68f54c518c0c8e931fed5941: Bootstrap starting.
I20240502 13:59:44.563783 25356 raft_consensus.cc:348] T cac3bbd142444b378b729de5cea6d1ae P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.563773 25355 raft_consensus.cc:348] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.563879 25356 raft_consensus.cc:374] T cac3bbd142444b378b729de5cea6d1ae P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.563879 25355 raft_consensus.cc:374] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.563916 25356 raft_consensus.cc:729] T cac3bbd142444b378b729de5cea6d1ae P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54fbce87fe064ab5a46db4b4197a093d, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.563916 25355 raft_consensus.cc:729] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c7ba84b0bb4626af59a8c46dcbb9c1, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.564033 25356 consensus_queue.cc:260] T cac3bbd142444b378b729de5cea6d1ae P 54fbce87fe064ab5a46db4b4197a093d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.564033 25355 consensus_queue.cc:260] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.564267 25356 ts_tablet_manager.cc:1398] T cac3bbd142444b378b729de5cea6d1ae P 54fbce87fe064ab5a46db4b4197a093d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:44.564267 25355 ts_tablet_manager.cc:1398] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:44.564443 25356 tablet_bootstrap.cc:492] T 1c75f6e566944a118ff217926b8b1f72 P 54fbce87fe064ab5a46db4b4197a093d: Bootstrap starting.
I20240502 13:59:44.564443 25355 tablet_bootstrap.cc:492] T 241895b448ae436b9b527a94c3331468 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Bootstrap starting.
I20240502 13:59:44.564711 25354 tablet_bootstrap.cc:654] T 2dbde6a2cb244667b7f24b6c4cb2c376 P f09f152a68f54c518c0c8e931fed5941: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.565495 25355 tablet_bootstrap.cc:654] T 241895b448ae436b9b527a94c3331468 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.565788 25354 tablet_bootstrap.cc:492] T 2dbde6a2cb244667b7f24b6c4cb2c376 P f09f152a68f54c518c0c8e931fed5941: No bootstrap required, opened a new log
I20240502 13:59:44.565862 25354 ts_tablet_manager.cc:1367] T 2dbde6a2cb244667b7f24b6c4cb2c376 P f09f152a68f54c518c0c8e931fed5941: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:44.566336 25356 tablet_bootstrap.cc:654] T 1c75f6e566944a118ff217926b8b1f72 P 54fbce87fe064ab5a46db4b4197a093d: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.566322 25354 raft_consensus.cc:348] T 2dbde6a2cb244667b7f24b6c4cb2c376 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.566421 25354 raft_consensus.cc:374] T 2dbde6a2cb244667b7f24b6c4cb2c376 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.566454 25354 raft_consensus.cc:729] T 2dbde6a2cb244667b7f24b6c4cb2c376 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f09f152a68f54c518c0c8e931fed5941, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.566560 25354 consensus_queue.cc:260] T 2dbde6a2cb244667b7f24b6c4cb2c376 P f09f152a68f54c518c0c8e931fed5941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.567113 25354 ts_tablet_manager.cc:1398] T 2dbde6a2cb244667b7f24b6c4cb2c376 P f09f152a68f54c518c0c8e931fed5941: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:44.567281 25354 tablet_bootstrap.cc:492] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P f09f152a68f54c518c0c8e931fed5941: Bootstrap starting.
I20240502 13:59:44.568094 25355 tablet_bootstrap.cc:492] T 241895b448ae436b9b527a94c3331468 P 45c7ba84b0bb4626af59a8c46dcbb9c1: No bootstrap required, opened a new log
I20240502 13:59:44.568166 25355 ts_tablet_manager.cc:1367] T 241895b448ae436b9b527a94c3331468 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 13:59:44.568226 25354 tablet_bootstrap.cc:654] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P f09f152a68f54c518c0c8e931fed5941: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.568645 25355 raft_consensus.cc:348] T 241895b448ae436b9b527a94c3331468 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.568739 25355 raft_consensus.cc:374] T 241895b448ae436b9b527a94c3331468 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.568774 25355 raft_consensus.cc:729] T 241895b448ae436b9b527a94c3331468 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c7ba84b0bb4626af59a8c46dcbb9c1, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.568897 25355 consensus_queue.cc:260] T 241895b448ae436b9b527a94c3331468 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.569160 25355 ts_tablet_manager.cc:1398] T 241895b448ae436b9b527a94c3331468 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:44.569314 25355 tablet_bootstrap.cc:492] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Bootstrap starting.
I20240502 13:59:44.569497 25356 tablet_bootstrap.cc:492] T 1c75f6e566944a118ff217926b8b1f72 P 54fbce87fe064ab5a46db4b4197a093d: No bootstrap required, opened a new log
I20240502 13:59:44.569497 25354 tablet_bootstrap.cc:492] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P f09f152a68f54c518c0c8e931fed5941: No bootstrap required, opened a new log
I20240502 13:59:44.569568 25354 ts_tablet_manager.cc:1367] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P f09f152a68f54c518c0c8e931fed5941: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:44.569568 25356 ts_tablet_manager.cc:1367] T 1c75f6e566944a118ff217926b8b1f72 P 54fbce87fe064ab5a46db4b4197a093d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 13:59:44.569974 25356 raft_consensus.cc:348] T 1c75f6e566944a118ff217926b8b1f72 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.569959 25354 raft_consensus.cc:348] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.570063 25354 raft_consensus.cc:374] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.570063 25356 raft_consensus.cc:374] T 1c75f6e566944a118ff217926b8b1f72 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.570096 25354 raft_consensus.cc:729] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f09f152a68f54c518c0c8e931fed5941, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.570096 25356 raft_consensus.cc:729] T 1c75f6e566944a118ff217926b8b1f72 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54fbce87fe064ab5a46db4b4197a093d, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.570212 25354 consensus_queue.cc:260] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P f09f152a68f54c518c0c8e931fed5941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.570264 25355 tablet_bootstrap.cc:654] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.570236 25356 consensus_queue.cc:260] T 1c75f6e566944a118ff217926b8b1f72 P 54fbce87fe064ab5a46db4b4197a093d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.570463 25356 ts_tablet_manager.cc:1398] T 1c75f6e566944a118ff217926b8b1f72 P 54fbce87fe064ab5a46db4b4197a093d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:44.570462 25354 ts_tablet_manager.cc:1398] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P f09f152a68f54c518c0c8e931fed5941: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:44.570613 25356 tablet_bootstrap.cc:492] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 54fbce87fe064ab5a46db4b4197a093d: Bootstrap starting.
I20240502 13:59:44.570619 25354 tablet_bootstrap.cc:492] T cac3bbd142444b378b729de5cea6d1ae P f09f152a68f54c518c0c8e931fed5941: Bootstrap starting.
I20240502 13:59:44.571557 25354 tablet_bootstrap.cc:654] T cac3bbd142444b378b729de5cea6d1ae P f09f152a68f54c518c0c8e931fed5941: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.571662 25356 tablet_bootstrap.cc:654] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 54fbce87fe064ab5a46db4b4197a093d: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.572849 25354 tablet_bootstrap.cc:492] T cac3bbd142444b378b729de5cea6d1ae P f09f152a68f54c518c0c8e931fed5941: No bootstrap required, opened a new log
I20240502 13:59:44.572934 25354 ts_tablet_manager.cc:1367] T cac3bbd142444b378b729de5cea6d1ae P f09f152a68f54c518c0c8e931fed5941: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:44.573120 25355 tablet_bootstrap.cc:492] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 45c7ba84b0bb4626af59a8c46dcbb9c1: No bootstrap required, opened a new log
I20240502 13:59:44.573190 25355 ts_tablet_manager.cc:1367] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 13:59:44.573385 25354 raft_consensus.cc:348] T cac3bbd142444b378b729de5cea6d1ae P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.573475 25354 raft_consensus.cc:374] T cac3bbd142444b378b729de5cea6d1ae P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.573508 25354 raft_consensus.cc:729] T cac3bbd142444b378b729de5cea6d1ae P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f09f152a68f54c518c0c8e931fed5941, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.573618 25354 consensus_queue.cc:260] T cac3bbd142444b378b729de5cea6d1ae P f09f152a68f54c518c0c8e931fed5941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.573642 25355 raft_consensus.cc:348] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.573726 25355 raft_consensus.cc:374] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.573756 25355 raft_consensus.cc:729] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c7ba84b0bb4626af59a8c46dcbb9c1, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.573808 25356 tablet_bootstrap.cc:492] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 54fbce87fe064ab5a46db4b4197a093d: No bootstrap required, opened a new log
I20240502 13:59:44.573894 25356 ts_tablet_manager.cc:1367] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 54fbce87fe064ab5a46db4b4197a093d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 13:59:44.573894 25354 ts_tablet_manager.cc:1398] T cac3bbd142444b378b729de5cea6d1ae P f09f152a68f54c518c0c8e931fed5941: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:44.573863 25355 consensus_queue.cc:260] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.574034 25354 tablet_bootstrap.cc:492] T d7b93d774c9d45858e14ed8fd6cc5bf2 P f09f152a68f54c518c0c8e931fed5941: Bootstrap starting.
I20240502 13:59:44.574123 25355 ts_tablet_manager.cc:1398] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:44.574263 25355 tablet_bootstrap.cc:492] T 1c75f6e566944a118ff217926b8b1f72 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Bootstrap starting.
I20240502 13:59:44.574330 25356 raft_consensus.cc:348] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.574433 25356 raft_consensus.cc:374] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.574466 25356 raft_consensus.cc:729] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54fbce87fe064ab5a46db4b4197a093d, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.574605 25356 consensus_queue.cc:260] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 54fbce87fe064ab5a46db4b4197a093d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.574884 25356 ts_tablet_manager.cc:1398] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 54fbce87fe064ab5a46db4b4197a093d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:44.574986 25354 tablet_bootstrap.cc:654] T d7b93d774c9d45858e14ed8fd6cc5bf2 P f09f152a68f54c518c0c8e931fed5941: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.575043 25356 tablet_bootstrap.cc:492] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 54fbce87fe064ab5a46db4b4197a093d: Bootstrap starting.
I20240502 13:59:44.575186 25355 tablet_bootstrap.cc:654] T 1c75f6e566944a118ff217926b8b1f72 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.575986 25356 tablet_bootstrap.cc:654] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 54fbce87fe064ab5a46db4b4197a093d: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.576277 25355 tablet_bootstrap.cc:492] T 1c75f6e566944a118ff217926b8b1f72 P 45c7ba84b0bb4626af59a8c46dcbb9c1: No bootstrap required, opened a new log
I20240502 13:59:44.576375 25355 ts_tablet_manager.cc:1367] T 1c75f6e566944a118ff217926b8b1f72 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:44.577361 25355 raft_consensus.cc:348] T 1c75f6e566944a118ff217926b8b1f72 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.577530 25355 raft_consensus.cc:374] T 1c75f6e566944a118ff217926b8b1f72 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.577569 25355 raft_consensus.cc:729] T 1c75f6e566944a118ff217926b8b1f72 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c7ba84b0bb4626af59a8c46dcbb9c1, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.577689 25354 tablet_bootstrap.cc:492] T d7b93d774c9d45858e14ed8fd6cc5bf2 P f09f152a68f54c518c0c8e931fed5941: No bootstrap required, opened a new log
I20240502 13:59:44.577684 25355 consensus_queue.cc:260] T 1c75f6e566944a118ff217926b8b1f72 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.577769 25354 ts_tablet_manager.cc:1367] T d7b93d774c9d45858e14ed8fd6cc5bf2 P f09f152a68f54c518c0c8e931fed5941: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 13:59:44.577950 25355 ts_tablet_manager.cc:1398] T 1c75f6e566944a118ff217926b8b1f72 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:44.578123 25355 tablet_bootstrap.cc:492] T 68513fdc8f5745088711a271afdbfe68 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Bootstrap starting.
I20240502 13:59:44.578197 25354 raft_consensus.cc:348] T d7b93d774c9d45858e14ed8fd6cc5bf2 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.578308 25354 raft_consensus.cc:374] T d7b93d774c9d45858e14ed8fd6cc5bf2 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.578339 25354 raft_consensus.cc:729] T d7b93d774c9d45858e14ed8fd6cc5bf2 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f09f152a68f54c518c0c8e931fed5941, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.578459 25354 consensus_queue.cc:260] T d7b93d774c9d45858e14ed8fd6cc5bf2 P f09f152a68f54c518c0c8e931fed5941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.578505 25356 tablet_bootstrap.cc:492] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 54fbce87fe064ab5a46db4b4197a093d: No bootstrap required, opened a new log
I20240502 13:59:44.578584 25356 ts_tablet_manager.cc:1367] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 54fbce87fe064ab5a46db4b4197a093d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240502 13:59:44.578724 25354 ts_tablet_manager.cc:1398] T d7b93d774c9d45858e14ed8fd6cc5bf2 P f09f152a68f54c518c0c8e931fed5941: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:44.578900 25354 tablet_bootstrap.cc:492] T 2b5e884f76de4e749872a7b9a2bd8493 P f09f152a68f54c518c0c8e931fed5941: Bootstrap starting.
I20240502 13:59:44.579030 25356 raft_consensus.cc:348] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.579146 25355 tablet_bootstrap.cc:654] T 68513fdc8f5745088711a271afdbfe68 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.579144 25356 raft_consensus.cc:374] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.579178 25356 raft_consensus.cc:729] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54fbce87fe064ab5a46db4b4197a093d, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.579298 25356 consensus_queue.cc:260] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 54fbce87fe064ab5a46db4b4197a093d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.579578 25356 ts_tablet_manager.cc:1398] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 54fbce87fe064ab5a46db4b4197a093d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:44.579767 25356 tablet_bootstrap.cc:492] T 2b5e884f76de4e749872a7b9a2bd8493 P 54fbce87fe064ab5a46db4b4197a093d: Bootstrap starting.
I20240502 13:59:44.579973 25354 tablet_bootstrap.cc:654] T 2b5e884f76de4e749872a7b9a2bd8493 P f09f152a68f54c518c0c8e931fed5941: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.580710 25356 tablet_bootstrap.cc:654] T 2b5e884f76de4e749872a7b9a2bd8493 P 54fbce87fe064ab5a46db4b4197a093d: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.580827 25355 tablet_bootstrap.cc:492] T 68513fdc8f5745088711a271afdbfe68 P 45c7ba84b0bb4626af59a8c46dcbb9c1: No bootstrap required, opened a new log
I20240502 13:59:44.580920 25355 ts_tablet_manager.cc:1367] T 68513fdc8f5745088711a271afdbfe68 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 13:59:44.581365 25355 raft_consensus.cc:348] T 68513fdc8f5745088711a271afdbfe68 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.581490 25355 raft_consensus.cc:374] T 68513fdc8f5745088711a271afdbfe68 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.581526 25355 raft_consensus.cc:729] T 68513fdc8f5745088711a271afdbfe68 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c7ba84b0bb4626af59a8c46dcbb9c1, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.581642 25355 consensus_queue.cc:260] T 68513fdc8f5745088711a271afdbfe68 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.581892 25355 ts_tablet_manager.cc:1398] T 68513fdc8f5745088711a271afdbfe68 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:44.582023 25354 tablet_bootstrap.cc:492] T 2b5e884f76de4e749872a7b9a2bd8493 P f09f152a68f54c518c0c8e931fed5941: No bootstrap required, opened a new log
I20240502 13:59:44.582060 25355 tablet_bootstrap.cc:492] T 65bd1232d19e4a2c903fa19ede36a9ba P 45c7ba84b0bb4626af59a8c46dcbb9c1: Bootstrap starting.
I20240502 13:59:44.582093 25354 ts_tablet_manager.cc:1367] T 2b5e884f76de4e749872a7b9a2bd8493 P f09f152a68f54c518c0c8e931fed5941: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 13:59:44.582520 25354 raft_consensus.cc:348] T 2b5e884f76de4e749872a7b9a2bd8493 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.582633 25354 raft_consensus.cc:374] T 2b5e884f76de4e749872a7b9a2bd8493 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.582653 25356 tablet_bootstrap.cc:492] T 2b5e884f76de4e749872a7b9a2bd8493 P 54fbce87fe064ab5a46db4b4197a093d: No bootstrap required, opened a new log
I20240502 13:59:44.582664 25354 raft_consensus.cc:729] T 2b5e884f76de4e749872a7b9a2bd8493 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f09f152a68f54c518c0c8e931fed5941, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.582728 25356 ts_tablet_manager.cc:1367] T 2b5e884f76de4e749872a7b9a2bd8493 P 54fbce87fe064ab5a46db4b4197a093d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 13:59:44.582782 25354 consensus_queue.cc:260] T 2b5e884f76de4e749872a7b9a2bd8493 P f09f152a68f54c518c0c8e931fed5941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.582998 25355 tablet_bootstrap.cc:654] T 65bd1232d19e4a2c903fa19ede36a9ba P 45c7ba84b0bb4626af59a8c46dcbb9c1: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.583052 25354 ts_tablet_manager.cc:1398] T 2b5e884f76de4e749872a7b9a2bd8493 P f09f152a68f54c518c0c8e931fed5941: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:44.583209 25354 tablet_bootstrap.cc:492] T b99e4b9853404fe188ec39b5d2a6c154 P f09f152a68f54c518c0c8e931fed5941: Bootstrap starting.
I20240502 13:59:44.583189 25356 raft_consensus.cc:348] T 2b5e884f76de4e749872a7b9a2bd8493 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.583284 25356 raft_consensus.cc:374] T 2b5e884f76de4e749872a7b9a2bd8493 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.583317 25356 raft_consensus.cc:729] T 2b5e884f76de4e749872a7b9a2bd8493 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54fbce87fe064ab5a46db4b4197a093d, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.583421 25356 consensus_queue.cc:260] T 2b5e884f76de4e749872a7b9a2bd8493 P 54fbce87fe064ab5a46db4b4197a093d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.583683 25356 ts_tablet_manager.cc:1398] T 2b5e884f76de4e749872a7b9a2bd8493 P 54fbce87fe064ab5a46db4b4197a093d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:44.583860 25356 tablet_bootstrap.cc:492] T 68513fdc8f5745088711a271afdbfe68 P 54fbce87fe064ab5a46db4b4197a093d: Bootstrap starting.
I20240502 13:59:44.584002 25361 raft_consensus.cc:482] T 1c75f6e566944a118ff217926b8b1f72 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:44.584182 25354 tablet_bootstrap.cc:654] T b99e4b9853404fe188ec39b5d2a6c154 P f09f152a68f54c518c0c8e931fed5941: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.584146 25361 raft_consensus.cc:504] T 1c75f6e566944a118ff217926b8b1f72 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.584758 25356 tablet_bootstrap.cc:654] T 68513fdc8f5745088711a271afdbfe68 P 54fbce87fe064ab5a46db4b4197a093d: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.585453 25361 leader_election.cc:288] T 1c75f6e566944a118ff217926b8b1f72 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65:34575), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.586199 25355 tablet_bootstrap.cc:492] T 65bd1232d19e4a2c903fa19ede36a9ba P 45c7ba84b0bb4626af59a8c46dcbb9c1: No bootstrap required, opened a new log
I20240502 13:59:44.586280 25355 ts_tablet_manager.cc:1367] T 65bd1232d19e4a2c903fa19ede36a9ba P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 13:59:44.586445 25354 tablet_bootstrap.cc:492] T b99e4b9853404fe188ec39b5d2a6c154 P f09f152a68f54c518c0c8e931fed5941: No bootstrap required, opened a new log
I20240502 13:59:44.586513 25354 ts_tablet_manager.cc:1367] T b99e4b9853404fe188ec39b5d2a6c154 P f09f152a68f54c518c0c8e931fed5941: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 13:59:44.586724 25355 raft_consensus.cc:348] T 65bd1232d19e4a2c903fa19ede36a9ba P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.586812 25355 raft_consensus.cc:374] T 65bd1232d19e4a2c903fa19ede36a9ba P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.586843 25355 raft_consensus.cc:729] T 65bd1232d19e4a2c903fa19ede36a9ba P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c7ba84b0bb4626af59a8c46dcbb9c1, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.586915 25354 raft_consensus.cc:348] T b99e4b9853404fe188ec39b5d2a6c154 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.586954 25355 consensus_queue.cc:260] T 65bd1232d19e4a2c903fa19ede36a9ba P 45c7ba84b0bb4626af59a8c46dcbb9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.586999 25354 raft_consensus.cc:374] T b99e4b9853404fe188ec39b5d2a6c154 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.587030 25354 raft_consensus.cc:729] T b99e4b9853404fe188ec39b5d2a6c154 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f09f152a68f54c518c0c8e931fed5941, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.587136 25354 consensus_queue.cc:260] T b99e4b9853404fe188ec39b5d2a6c154 P f09f152a68f54c518c0c8e931fed5941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.587215 25355 ts_tablet_manager.cc:1398] T 65bd1232d19e4a2c903fa19ede36a9ba P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:44.587364 25355 tablet_bootstrap.cc:492] T b99e4b9853404fe188ec39b5d2a6c154 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Bootstrap starting.
I20240502 13:59:44.587425 25354 ts_tablet_manager.cc:1398] T b99e4b9853404fe188ec39b5d2a6c154 P f09f152a68f54c518c0c8e931fed5941: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 13:59:44.587553 25354 tablet_bootstrap.cc:492] T adc3311b364245c69107c0cef3c5ce8a P f09f152a68f54c518c0c8e931fed5941: Bootstrap starting.
I20240502 13:59:44.588444 25354 tablet_bootstrap.cc:654] T adc3311b364245c69107c0cef3c5ce8a P f09f152a68f54c518c0c8e931fed5941: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.588876 25361 raft_consensus.cc:482] T cac3bbd142444b378b729de5cea6d1ae P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:44.588979 25361 raft_consensus.cc:504] T cac3bbd142444b378b729de5cea6d1ae P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.589334 25361 leader_election.cc:288] T cac3bbd142444b378b729de5cea6d1ae P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65:34575), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.589485 25355 tablet_bootstrap.cc:654] T b99e4b9853404fe188ec39b5d2a6c154 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.589668 25361 raft_consensus.cc:482] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:44.589701 25360 raft_consensus.cc:482] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:44.589753 25361 raft_consensus.cc:504] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.589860 25360 raft_consensus.cc:504] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.590047 25361 leader_election.cc:288] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65:34575), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.590585 25356 tablet_bootstrap.cc:492] T 68513fdc8f5745088711a271afdbfe68 P 54fbce87fe064ab5a46db4b4197a093d: No bootstrap required, opened a new log
I20240502 13:59:44.590721 25356 ts_tablet_manager.cc:1367] T 68513fdc8f5745088711a271afdbfe68 P 54fbce87fe064ab5a46db4b4197a093d: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 13:59:44.591015 25354 tablet_bootstrap.cc:492] T adc3311b364245c69107c0cef3c5ce8a P f09f152a68f54c518c0c8e931fed5941: No bootstrap required, opened a new log
I20240502 13:59:44.591090 25354 ts_tablet_manager.cc:1367] T adc3311b364245c69107c0cef3c5ce8a P f09f152a68f54c518c0c8e931fed5941: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 13:59:44.591143 25360 leader_election.cc:288] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66:40803), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.591557 25354 raft_consensus.cc:348] T adc3311b364245c69107c0cef3c5ce8a P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.591709 25354 raft_consensus.cc:374] T adc3311b364245c69107c0cef3c5ce8a P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.591784 25354 raft_consensus.cc:729] T adc3311b364245c69107c0cef3c5ce8a P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f09f152a68f54c518c0c8e931fed5941, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.591980 25354 consensus_queue.cc:260] T adc3311b364245c69107c0cef3c5ce8a P f09f152a68f54c518c0c8e931fed5941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.592231 25355 tablet_bootstrap.cc:492] T b99e4b9853404fe188ec39b5d2a6c154 P 45c7ba84b0bb4626af59a8c46dcbb9c1: No bootstrap required, opened a new log
I20240502 13:59:44.592357 25355 ts_tablet_manager.cc:1367] T b99e4b9853404fe188ec39b5d2a6c154 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 13:59:44.592368 25354 ts_tablet_manager.cc:1398] T adc3311b364245c69107c0cef3c5ce8a P f09f152a68f54c518c0c8e931fed5941: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 13:59:44.592532 25354 tablet_bootstrap.cc:492] T c8a7d300caa44b56918dc7045722e3c1 P f09f152a68f54c518c0c8e931fed5941: Bootstrap starting.
I20240502 13:59:44.592836 25355 raft_consensus.cc:348] T b99e4b9853404fe188ec39b5d2a6c154 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.592926 25355 raft_consensus.cc:374] T b99e4b9853404fe188ec39b5d2a6c154 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.592952 25355 raft_consensus.cc:729] T b99e4b9853404fe188ec39b5d2a6c154 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c7ba84b0bb4626af59a8c46dcbb9c1, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.593051 25355 consensus_queue.cc:260] T b99e4b9853404fe188ec39b5d2a6c154 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.593317 25355 ts_tablet_manager.cc:1398] T b99e4b9853404fe188ec39b5d2a6c154 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 13:59:44.593463 25355 tablet_bootstrap.cc:492] T 34007af599ef4691977c361f01f22767 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Bootstrap starting.
W20240502 13:59:44.593853 25075 tablet.cc:2369] T 0bd322784404440983b3ca1026f13c10 P 54fbce87fe064ab5a46db4b4197a093d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 13:59:44.594269 25360 raft_consensus.cc:482] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:44.594345 25360 raft_consensus.cc:504] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.594594 25355 tablet_bootstrap.cc:654] T 34007af599ef4691977c361f01f22767 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.594723 25360 leader_election.cc:288] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66:40803), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.595237 25028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c75f6e566944a118ff217926b8b1f72" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54fbce87fe064ab5a46db4b4197a093d" is_pre_election: true
I20240502 13:59:44.595438 25028 raft_consensus.cc:2454] T 1c75f6e566944a118ff217926b8b1f72 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 0.
I20240502 13:59:44.595530 25354 tablet_bootstrap.cc:654] T c8a7d300caa44b56918dc7045722e3c1 P f09f152a68f54c518c0c8e931fed5941: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.595759 25027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cac3bbd142444b378b729de5cea6d1ae" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54fbce87fe064ab5a46db4b4197a093d" is_pre_election: true
I20240502 13:59:44.595865 25027 raft_consensus.cc:2454] T cac3bbd142444b378b729de5cea6d1ae P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 0.
I20240502 13:59:44.595930 25094 leader_election.cc:302] T 1c75f6e566944a118ff217926b8b1f72 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45c7ba84b0bb4626af59a8c46dcbb9c1, 54fbce87fe064ab5a46db4b4197a093d; no voters:
I20240502 13:59:44.595875 25026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b93d774c9d45858e14ed8fd6cc5bf2" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54fbce87fe064ab5a46db4b4197a093d" is_pre_election: true
I20240502 13:59:44.596112 25026 raft_consensus.cc:2454] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 0.
I20240502 13:59:44.596201 25361 raft_consensus.cc:2781] T 1c75f6e566944a118ff217926b8b1f72 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:44.596278 25361 raft_consensus.cc:482] T 1c75f6e566944a118ff217926b8b1f72 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:44.596350 25361 raft_consensus.cc:3037] T 1c75f6e566944a118ff217926b8b1f72 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.596616 25094 leader_election.cc:302] T cac3bbd142444b378b729de5cea6d1ae P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45c7ba84b0bb4626af59a8c46dcbb9c1, 54fbce87fe064ab5a46db4b4197a093d; no voters:
I20240502 13:59:44.596796 25094 leader_election.cc:302] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45c7ba84b0bb4626af59a8c46dcbb9c1, 54fbce87fe064ab5a46db4b4197a093d; no voters:
I20240502 13:59:44.597155 25391 raft_consensus.cc:2781] T cac3bbd142444b378b729de5cea6d1ae P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:44.597213 25391 raft_consensus.cc:482] T cac3bbd142444b378b729de5cea6d1ae P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:44.597250 25391 raft_consensus.cc:3037] T cac3bbd142444b378b729de5cea6d1ae P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.597543 25361 raft_consensus.cc:504] T 1c75f6e566944a118ff217926b8b1f72 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.597895 25361 leader_election.cc:288] T 1c75f6e566944a118ff217926b8b1f72 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 election: Requested vote from peers 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65:34575), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.597988 25361 raft_consensus.cc:2781] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:44.598026 25361 raft_consensus.cc:482] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:44.598060 25361 raft_consensus.cc:3037] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.591141 25356 raft_consensus.cc:348] T 68513fdc8f5745088711a271afdbfe68 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.599063 25356 raft_consensus.cc:374] T 68513fdc8f5745088711a271afdbfe68 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.599104 25356 raft_consensus.cc:729] T 68513fdc8f5745088711a271afdbfe68 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54fbce87fe064ab5a46db4b4197a093d, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.599267 25356 consensus_queue.cc:260] T 68513fdc8f5745088711a271afdbfe68 P 54fbce87fe064ab5a46db4b4197a093d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.599613 25356 ts_tablet_manager.cc:1398] T 68513fdc8f5745088711a271afdbfe68 P 54fbce87fe064ab5a46db4b4197a093d: Time spent starting tablet: real 0.009s user 0.001s sys 0.000s
I20240502 13:59:44.599817 25356 tablet_bootstrap.cc:492] T 5617b9f3b6bb436e88a2aeaf051c4700 P 54fbce87fe064ab5a46db4b4197a093d: Bootstrap starting.
I20240502 13:59:44.599857 25026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c75f6e566944a118ff217926b8b1f72" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54fbce87fe064ab5a46db4b4197a093d"
I20240502 13:59:44.599956 25026 raft_consensus.cc:3037] T 1c75f6e566944a118ff217926b8b1f72 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.601502 25026 raft_consensus.cc:2454] T 1c75f6e566944a118ff217926b8b1f72 P 54fbce87fe064ab5a46db4b4197a093d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 1.
I20240502 13:59:44.601600 25290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99e4b9853404fe188ec39b5d2a6c154" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941" is_pre_election: true
I20240502 13:59:44.601820 25355 tablet_bootstrap.cc:492] T 34007af599ef4691977c361f01f22767 P 45c7ba84b0bb4626af59a8c46dcbb9c1: No bootstrap required, opened a new log
I20240502 13:59:44.601864 25290 raft_consensus.cc:2454] T b99e4b9853404fe188ec39b5d2a6c154 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 0.
I20240502 13:59:44.601920 25355 ts_tablet_manager.cc:1367] T 34007af599ef4691977c361f01f22767 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20240502 13:59:44.601944 25094 leader_election.cc:302] T 1c75f6e566944a118ff217926b8b1f72 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45c7ba84b0bb4626af59a8c46dcbb9c1, 54fbce87fe064ab5a46db4b4197a093d; no voters:
I20240502 13:59:44.602147 25289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dbde6a2cb244667b7f24b6c4cb2c376" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941" is_pre_election: true
I20240502 13:59:44.602253 25289 raft_consensus.cc:2454] T 2dbde6a2cb244667b7f24b6c4cb2c376 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 0.
I20240502 13:59:44.602411 25355 raft_consensus.cc:348] T 34007af599ef4691977c361f01f22767 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.602521 25355 raft_consensus.cc:374] T 34007af599ef4691977c361f01f22767 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.602533 25356 tablet_bootstrap.cc:654] T 5617b9f3b6bb436e88a2aeaf051c4700 P 54fbce87fe064ab5a46db4b4197a093d: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.602561 25355 raft_consensus.cc:729] T 34007af599ef4691977c361f01f22767 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c7ba84b0bb4626af59a8c46dcbb9c1, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.602588 24962 leader_election.cc:302] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54fbce87fe064ab5a46db4b4197a093d, f09f152a68f54c518c0c8e931fed5941; no voters:
I20240502 13:59:44.602681 25355 consensus_queue.cc:260] T 34007af599ef4691977c361f01f22767 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.602818 25360 raft_consensus.cc:2781] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:44.602813 25361 raft_consensus.cc:504] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.602871 25360 raft_consensus.cc:482] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:44.602942 25360 raft_consensus.cc:3037] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.602860 24962 leader_election.cc:302] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54fbce87fe064ab5a46db4b4197a093d, f09f152a68f54c518c0c8e931fed5941; no voters:
I20240502 13:59:44.603076 25355 ts_tablet_manager.cc:1398] T 34007af599ef4691977c361f01f22767 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 13:59:44.603123 25361 leader_election.cc:288] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 election: Requested vote from peers 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65:34575), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.603199 25361 raft_consensus.cc:2781] T 1c75f6e566944a118ff217926b8b1f72 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:44.603260 25361 raft_consensus.cc:686] T 1c75f6e566944a118ff217926b8b1f72 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 LEADER]: Becoming Leader. State: Replica: 45c7ba84b0bb4626af59a8c46dcbb9c1, State: Running, Role: LEADER
I20240502 13:59:44.603248 25355 tablet_bootstrap.cc:492] T 5617b9f3b6bb436e88a2aeaf051c4700 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Bootstrap starting.
I20240502 13:59:44.603583 25361 consensus_queue.cc:237] T 1c75f6e566944a118ff217926b8b1f72 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.603879 25360 raft_consensus.cc:504] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.604072 25405 raft_consensus.cc:2781] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:44.604131 25405 raft_consensus.cc:482] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:44.604176 25405 raft_consensus.cc:3037] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.604383 25360 leader_election.cc:288] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 election: Requested vote from peers 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66:40803), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.604717 25289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99e4b9853404fe188ec39b5d2a6c154" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941"
I20240502 13:59:44.604828 25289 raft_consensus.cc:3037] T b99e4b9853404fe188ec39b5d2a6c154 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.605073 25391 raft_consensus.cc:504] T cac3bbd142444b378b729de5cea6d1ae P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.605337 25026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b93d774c9d45858e14ed8fd6cc5bf2" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54fbce87fe064ab5a46db4b4197a093d"
I20240502 13:59:44.605429 25026 raft_consensus.cc:3037] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.605810 25391 leader_election.cc:288] T cac3bbd142444b378b729de5cea6d1ae P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 election: Requested vote from peers 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65:34575), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.605995 25289 raft_consensus.cc:2454] T b99e4b9853404fe188ec39b5d2a6c154 P f09f152a68f54c518c0c8e931fed5941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 1.
I20240502 13:59:44.606251 25355 tablet_bootstrap.cc:654] T 5617b9f3b6bb436e88a2aeaf051c4700 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.606256 25026 raft_consensus.cc:2454] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 54fbce87fe064ab5a46db4b4197a093d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 1.
I20240502 13:59:44.606549 24962 leader_election.cc:302] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54fbce87fe064ab5a46db4b4197a093d, f09f152a68f54c518c0c8e931fed5941; no voters:
I20240502 13:59:44.606952 25360 raft_consensus.cc:2781] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:44.607144 25094 leader_election.cc:302] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45c7ba84b0bb4626af59a8c46dcbb9c1, 54fbce87fe064ab5a46db4b4197a093d; no voters:
I20240502 13:59:44.607302 25391 raft_consensus.cc:2781] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:44.607395 25391 raft_consensus.cc:686] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 LEADER]: Becoming Leader. State: Replica: 45c7ba84b0bb4626af59a8c46dcbb9c1, State: Running, Role: LEADER
I20240502 13:59:44.607523 25391 consensus_queue.cc:237] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.608263 25026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cac3bbd142444b378b729de5cea6d1ae" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54fbce87fe064ab5a46db4b4197a093d"
I20240502 13:59:44.608317 25360 raft_consensus.cc:686] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d [term 1 LEADER]: Becoming Leader. State: Replica: 54fbce87fe064ab5a46db4b4197a093d, State: Running, Role: LEADER
I20240502 13:59:44.608424 25026 raft_consensus.cc:3037] T cac3bbd142444b378b729de5cea6d1ae P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.609412 25026 raft_consensus.cc:2454] T cac3bbd142444b378b729de5cea6d1ae P 54fbce87fe064ab5a46db4b4197a093d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 1.
I20240502 13:59:44.609516 25356 tablet_bootstrap.cc:492] T 5617b9f3b6bb436e88a2aeaf051c4700 P 54fbce87fe064ab5a46db4b4197a093d: No bootstrap required, opened a new log
I20240502 13:59:44.609612 25356 ts_tablet_manager.cc:1367] T 5617b9f3b6bb436e88a2aeaf051c4700 P 54fbce87fe064ab5a46db4b4197a093d: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20240502 13:59:44.609614 25355 tablet_bootstrap.cc:492] T 5617b9f3b6bb436e88a2aeaf051c4700 P 45c7ba84b0bb4626af59a8c46dcbb9c1: No bootstrap required, opened a new log
I20240502 13:59:44.609694 25355 ts_tablet_manager.cc:1367] T 5617b9f3b6bb436e88a2aeaf051c4700 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 13:59:44.608712 25360 consensus_queue.cc:237] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.609840 25094 leader_election.cc:302] T cac3bbd142444b378b729de5cea6d1ae P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45c7ba84b0bb4626af59a8c46dcbb9c1, 54fbce87fe064ab5a46db4b4197a093d; no voters:
I20240502 13:59:44.610025 25361 raft_consensus.cc:2781] T cac3bbd142444b378b729de5cea6d1ae P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:44.610096 25361 raft_consensus.cc:686] T cac3bbd142444b378b729de5cea6d1ae P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 LEADER]: Becoming Leader. State: Replica: 45c7ba84b0bb4626af59a8c46dcbb9c1, State: Running, Role: LEADER
I20240502 13:59:44.610200 25361 consensus_queue.cc:237] T cac3bbd142444b378b729de5cea6d1ae P 45c7ba84b0bb4626af59a8c46dcbb9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.610323 25356 raft_consensus.cc:348] T 5617b9f3b6bb436e88a2aeaf051c4700 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.610405 25355 raft_consensus.cc:348] T 5617b9f3b6bb436e88a2aeaf051c4700 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.610460 25356 raft_consensus.cc:374] T 5617b9f3b6bb436e88a2aeaf051c4700 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.610502 25356 raft_consensus.cc:729] T 5617b9f3b6bb436e88a2aeaf051c4700 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54fbce87fe064ab5a46db4b4197a093d, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.610488 25355 raft_consensus.cc:374] T 5617b9f3b6bb436e88a2aeaf051c4700 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.610535 25355 raft_consensus.cc:729] T 5617b9f3b6bb436e88a2aeaf051c4700 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c7ba84b0bb4626af59a8c46dcbb9c1, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.610603 25356 consensus_queue.cc:260] T 5617b9f3b6bb436e88a2aeaf051c4700 P 54fbce87fe064ab5a46db4b4197a093d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.610616 25355 consensus_queue.cc:260] T 5617b9f3b6bb436e88a2aeaf051c4700 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.610903 25355 ts_tablet_manager.cc:1398] T 5617b9f3b6bb436e88a2aeaf051c4700 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:44.610951 25356 ts_tablet_manager.cc:1398] T 5617b9f3b6bb436e88a2aeaf051c4700 P 54fbce87fe064ab5a46db4b4197a093d: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 13:59:44.611068 25355 tablet_bootstrap.cc:492] T c8a7d300caa44b56918dc7045722e3c1 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Bootstrap starting.
I20240502 13:59:44.611119 25356 tablet_bootstrap.cc:492] T adc3311b364245c69107c0cef3c5ce8a P 54fbce87fe064ab5a46db4b4197a093d: Bootstrap starting.
I20240502 13:59:44.611743 24882 catalog_manager.cc:5484] T 1c75f6e566944a118ff217926b8b1f72 P 45c7ba84b0bb4626af59a8c46dcbb9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66). New cstate: current_term: 1 leader_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } health_report { overall_health: UNKNOWN } } }
I20240502 13:59:44.612049 25355 tablet_bootstrap.cc:654] T c8a7d300caa44b56918dc7045722e3c1 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.612053 25356 tablet_bootstrap.cc:654] T adc3311b364245c69107c0cef3c5ce8a P 54fbce87fe064ab5a46db4b4197a093d: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.613029 25405 raft_consensus.cc:504] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 54fbce87fe064ab5a46db4b4197a093d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.613732 25289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dbde6a2cb244667b7f24b6c4cb2c376" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941"
I20240502 13:59:44.613858 25289 raft_consensus.cc:3037] T 2dbde6a2cb244667b7f24b6c4cb2c376 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.613826 24884 catalog_manager.cc:5484] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d reported cstate change: term changed from 0 to 1, leader changed from <none> to 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65). New cstate: current_term: 1 leader_uuid: "54fbce87fe064ab5a46db4b4197a093d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } health_report { overall_health: UNKNOWN } } }
I20240502 13:59:44.614812 25289 raft_consensus.cc:2454] T 2dbde6a2cb244667b7f24b6c4cb2c376 P f09f152a68f54c518c0c8e931fed5941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 1.
I20240502 13:59:44.615088 24962 leader_election.cc:302] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54fbce87fe064ab5a46db4b4197a093d, f09f152a68f54c518c0c8e931fed5941; no voters:
I20240502 13:59:44.615252 25405 leader_election.cc:288] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 election: Requested vote from peers 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66:40803), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.615258 25289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c75f6e566944a118ff217926b8b1f72" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941" is_pre_election: true
I20240502 13:59:44.615379 25289 raft_consensus.cc:2454] T 1c75f6e566944a118ff217926b8b1f72 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 0.
I20240502 13:59:44.615355 25408 raft_consensus.cc:2781] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 54fbce87fe064ab5a46db4b4197a093d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:44.615463 25408 raft_consensus.cc:686] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 54fbce87fe064ab5a46db4b4197a093d [term 1 LEADER]: Becoming Leader. State: Replica: 54fbce87fe064ab5a46db4b4197a093d, State: Running, Role: LEADER
I20240502 13:59:44.615537 25290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cac3bbd142444b378b729de5cea6d1ae" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941" is_pre_election: true
I20240502 13:59:44.615610 25288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b93d774c9d45858e14ed8fd6cc5bf2" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941" is_pre_election: true
I20240502 13:59:44.615645 25290 raft_consensus.cc:2454] T cac3bbd142444b378b729de5cea6d1ae P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 0.
I20240502 13:59:44.615617 25408 consensus_queue.cc:237] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 54fbce87fe064ab5a46db4b4197a093d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.615711 25288 raft_consensus.cc:2454] T d7b93d774c9d45858e14ed8fd6cc5bf2 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 0.
I20240502 13:59:44.615901 25287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c75f6e566944a118ff217926b8b1f72" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941"
I20240502 13:59:44.615995 25287 raft_consensus.cc:3037] T 1c75f6e566944a118ff217926b8b1f72 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.616495 25285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cac3bbd142444b378b729de5cea6d1ae" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941"
I20240502 13:59:44.616598 25285 raft_consensus.cc:3037] T cac3bbd142444b378b729de5cea6d1ae P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.616986 25287 raft_consensus.cc:2454] T 1c75f6e566944a118ff217926b8b1f72 P f09f152a68f54c518c0c8e931fed5941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 1.
I20240502 13:59:44.617292 25285 raft_consensus.cc:2454] T cac3bbd142444b378b729de5cea6d1ae P f09f152a68f54c518c0c8e931fed5941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 1.
I20240502 13:59:44.617754 25286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b93d774c9d45858e14ed8fd6cc5bf2" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941"
I20240502 13:59:44.617861 25286 raft_consensus.cc:3037] T d7b93d774c9d45858e14ed8fd6cc5bf2 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.617779 24884 catalog_manager.cc:5484] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 54fbce87fe064ab5a46db4b4197a093d reported cstate change: term changed from 0 to 1, leader changed from <none> to 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65). New cstate: current_term: 1 leader_uuid: "54fbce87fe064ab5a46db4b4197a093d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } health_report { overall_health: UNKNOWN } } }
I20240502 13:59:44.618667 25286 raft_consensus.cc:2454] T d7b93d774c9d45858e14ed8fd6cc5bf2 P f09f152a68f54c518c0c8e931fed5941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 1.
I20240502 13:59:44.620216 25159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99e4b9853404fe188ec39b5d2a6c154" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" is_pre_election: true
I20240502 13:59:44.620447 25158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dbde6a2cb244667b7f24b6c4cb2c376" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" is_pre_election: true
I20240502 13:59:44.620595 25158 raft_consensus.cc:2454] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 0.
I20240502 13:59:44.620869 25157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b99e4b9853404fe188ec39b5d2a6c154" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1"
I20240502 13:59:44.620959 25157 raft_consensus.cc:3037] T b99e4b9853404fe188ec39b5d2a6c154 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.621229 24884 catalog_manager.cc:5484] T cac3bbd142444b378b729de5cea6d1ae P 45c7ba84b0bb4626af59a8c46dcbb9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66). New cstate: current_term: 1 leader_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } health_report { overall_health: UNKNOWN } } }
I20240502 13:59:44.621472 24884 catalog_manager.cc:5484] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 45c7ba84b0bb4626af59a8c46dcbb9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66). New cstate: current_term: 1 leader_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } health_report { overall_health: UNKNOWN } } }
I20240502 13:59:44.621564 25356 tablet_bootstrap.cc:492] T adc3311b364245c69107c0cef3c5ce8a P 54fbce87fe064ab5a46db4b4197a093d: No bootstrap required, opened a new log
I20240502 13:59:44.621654 25356 ts_tablet_manager.cc:1367] T adc3311b364245c69107c0cef3c5ce8a P 54fbce87fe064ab5a46db4b4197a093d: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20240502 13:59:44.621966 25157 raft_consensus.cc:2454] T b99e4b9853404fe188ec39b5d2a6c154 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 1.
I20240502 13:59:44.622162 25156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dbde6a2cb244667b7f24b6c4cb2c376" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1"
I20240502 13:59:44.622159 25356 raft_consensus.cc:348] T adc3311b364245c69107c0cef3c5ce8a P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.622249 25156 raft_consensus.cc:3037] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.622269 25356 raft_consensus.cc:374] T adc3311b364245c69107c0cef3c5ce8a P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.622303 25356 raft_consensus.cc:729] T adc3311b364245c69107c0cef3c5ce8a P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54fbce87fe064ab5a46db4b4197a093d, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.622432 25356 consensus_queue.cc:260] T adc3311b364245c69107c0cef3c5ce8a P 54fbce87fe064ab5a46db4b4197a093d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.622740 25356 ts_tablet_manager.cc:1398] T adc3311b364245c69107c0cef3c5ce8a P 54fbce87fe064ab5a46db4b4197a093d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 13:59:44.622941 25356 tablet_bootstrap.cc:492] T 65bd1232d19e4a2c903fa19ede36a9ba P 54fbce87fe064ab5a46db4b4197a093d: Bootstrap starting.
I20240502 13:59:44.623101 25156 raft_consensus.cc:2454] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 1.
I20240502 13:59:44.623251 25354 tablet_bootstrap.cc:492] T c8a7d300caa44b56918dc7045722e3c1 P f09f152a68f54c518c0c8e931fed5941: No bootstrap required, opened a new log
I20240502 13:59:44.623337 25354 ts_tablet_manager.cc:1367] T c8a7d300caa44b56918dc7045722e3c1 P f09f152a68f54c518c0c8e931fed5941: Time spent bootstrapping tablet: real 0.031s user 0.000s sys 0.002s
I20240502 13:59:44.623375 25355 tablet_bootstrap.cc:492] T c8a7d300caa44b56918dc7045722e3c1 P 45c7ba84b0bb4626af59a8c46dcbb9c1: No bootstrap required, opened a new log
I20240502 13:59:44.623466 25355 ts_tablet_manager.cc:1367] T c8a7d300caa44b56918dc7045722e3c1 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 13:59:44.623822 25354 raft_consensus.cc:348] T c8a7d300caa44b56918dc7045722e3c1 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.623921 25354 raft_consensus.cc:374] T c8a7d300caa44b56918dc7045722e3c1 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.623945 25356 tablet_bootstrap.cc:654] T 65bd1232d19e4a2c903fa19ede36a9ba P 54fbce87fe064ab5a46db4b4197a093d: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.623955 25354 raft_consensus.cc:729] T c8a7d300caa44b56918dc7045722e3c1 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f09f152a68f54c518c0c8e931fed5941, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.623944 25355 raft_consensus.cc:348] T c8a7d300caa44b56918dc7045722e3c1 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.624040 25355 raft_consensus.cc:374] T c8a7d300caa44b56918dc7045722e3c1 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.624074 25355 raft_consensus.cc:729] T c8a7d300caa44b56918dc7045722e3c1 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c7ba84b0bb4626af59a8c46dcbb9c1, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.624080 25354 consensus_queue.cc:260] T c8a7d300caa44b56918dc7045722e3c1 P f09f152a68f54c518c0c8e931fed5941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.624191 25159 raft_consensus.cc:2362] T b99e4b9853404fe188ec39b5d2a6c154 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 1. Re-sending same reply.
I20240502 13:59:44.624198 25355 consensus_queue.cc:260] T c8a7d300caa44b56918dc7045722e3c1 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.624403 25354 ts_tablet_manager.cc:1398] T c8a7d300caa44b56918dc7045722e3c1 P f09f152a68f54c518c0c8e931fed5941: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 13:59:44.624565 25355 ts_tablet_manager.cc:1398] T c8a7d300caa44b56918dc7045722e3c1 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:44.624577 25354 tablet_bootstrap.cc:492] T 758640af315340bd8d9e919888d30c86 P f09f152a68f54c518c0c8e931fed5941: Bootstrap starting.
I20240502 13:59:44.624697 25412 raft_consensus.cc:482] T 68513fdc8f5745088711a271afdbfe68 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:44.624722 25355 tablet_bootstrap.cc:492] T 758640af315340bd8d9e919888d30c86 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Bootstrap starting.
I20240502 13:59:44.624774 25412 raft_consensus.cc:504] T 68513fdc8f5745088711a271afdbfe68 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.625114 25412 leader_election.cc:288] T 68513fdc8f5745088711a271afdbfe68 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66:40803), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.625451 25354 tablet_bootstrap.cc:654] T 758640af315340bd8d9e919888d30c86 P f09f152a68f54c518c0c8e931fed5941: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.625511 25355 tablet_bootstrap.cc:654] T 758640af315340bd8d9e919888d30c86 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.626341 25286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68513fdc8f5745088711a271afdbfe68" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941" is_pre_election: true
I20240502 13:59:44.626478 25286 raft_consensus.cc:2454] T 68513fdc8f5745088711a271afdbfe68 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 0.
I20240502 13:59:44.626734 24962 leader_election.cc:302] T 68513fdc8f5745088711a271afdbfe68 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54fbce87fe064ab5a46db4b4197a093d, f09f152a68f54c518c0c8e931fed5941; no voters:
I20240502 13:59:44.626892 25408 raft_consensus.cc:2781] T 68513fdc8f5745088711a271afdbfe68 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:44.626938 25408 raft_consensus.cc:482] T 68513fdc8f5745088711a271afdbfe68 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:44.626981 25408 raft_consensus.cc:3037] T 68513fdc8f5745088711a271afdbfe68 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.627050 25159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68513fdc8f5745088711a271afdbfe68" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" is_pre_election: true
I20240502 13:59:44.627189 25159 raft_consensus.cc:2454] T 68513fdc8f5745088711a271afdbfe68 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 0.
I20240502 13:59:44.628033 25408 raft_consensus.cc:504] T 68513fdc8f5745088711a271afdbfe68 P 54fbce87fe064ab5a46db4b4197a093d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.628405 25408 leader_election.cc:288] T 68513fdc8f5745088711a271afdbfe68 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 election: Requested vote from peers 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66:40803), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.628516 25354 tablet_bootstrap.cc:492] T 758640af315340bd8d9e919888d30c86 P f09f152a68f54c518c0c8e931fed5941: No bootstrap required, opened a new log
I20240502 13:59:44.628609 25354 ts_tablet_manager.cc:1367] T 758640af315340bd8d9e919888d30c86 P f09f152a68f54c518c0c8e931fed5941: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 13:59:44.628677 25286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68513fdc8f5745088711a271afdbfe68" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941"
I20240502 13:59:44.628772 25286 raft_consensus.cc:3037] T 68513fdc8f5745088711a271afdbfe68 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.628839 25159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68513fdc8f5745088711a271afdbfe68" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1"
I20240502 13:59:44.628927 25159 raft_consensus.cc:3037] T 68513fdc8f5745088711a271afdbfe68 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.628991 25354 raft_consensus.cc:348] T 758640af315340bd8d9e919888d30c86 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.629060 25354 raft_consensus.cc:374] T 758640af315340bd8d9e919888d30c86 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.629079 25354 raft_consensus.cc:729] T 758640af315340bd8d9e919888d30c86 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f09f152a68f54c518c0c8e931fed5941, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.629163 25354 consensus_queue.cc:260] T 758640af315340bd8d9e919888d30c86 P f09f152a68f54c518c0c8e931fed5941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.629374 25354 ts_tablet_manager.cc:1398] T 758640af315340bd8d9e919888d30c86 P f09f152a68f54c518c0c8e931fed5941: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:44.629490 25354 tablet_bootstrap.cc:492] T 5617b9f3b6bb436e88a2aeaf051c4700 P f09f152a68f54c518c0c8e931fed5941: Bootstrap starting.
I20240502 13:59:44.629648 25286 raft_consensus.cc:2454] T 68513fdc8f5745088711a271afdbfe68 P f09f152a68f54c518c0c8e931fed5941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 1.
I20240502 13:59:44.629761 25159 raft_consensus.cc:2454] T 68513fdc8f5745088711a271afdbfe68 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 1.
I20240502 13:59:44.629951 24962 leader_election.cc:302] T 68513fdc8f5745088711a271afdbfe68 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54fbce87fe064ab5a46db4b4197a093d, f09f152a68f54c518c0c8e931fed5941; no voters:
I20240502 13:59:44.630105 25408 raft_consensus.cc:2781] T 68513fdc8f5745088711a271afdbfe68 P 54fbce87fe064ab5a46db4b4197a093d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:44.630198 25408 raft_consensus.cc:686] T 68513fdc8f5745088711a271afdbfe68 P 54fbce87fe064ab5a46db4b4197a093d [term 1 LEADER]: Becoming Leader. State: Replica: 54fbce87fe064ab5a46db4b4197a093d, State: Running, Role: LEADER
I20240502 13:59:44.630303 25354 tablet_bootstrap.cc:654] T 5617b9f3b6bb436e88a2aeaf051c4700 P f09f152a68f54c518c0c8e931fed5941: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.630340 25408 consensus_queue.cc:237] T 68513fdc8f5745088711a271afdbfe68 P 54fbce87fe064ab5a46db4b4197a093d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.631897 25355 tablet_bootstrap.cc:492] T 758640af315340bd8d9e919888d30c86 P 45c7ba84b0bb4626af59a8c46dcbb9c1: No bootstrap required, opened a new log
I20240502 13:59:44.631981 25355 ts_tablet_manager.cc:1367] T 758640af315340bd8d9e919888d30c86 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 13:59:44.632208 25356 tablet_bootstrap.cc:492] T 65bd1232d19e4a2c903fa19ede36a9ba P 54fbce87fe064ab5a46db4b4197a093d: No bootstrap required, opened a new log
I20240502 13:59:44.632289 25356 ts_tablet_manager.cc:1367] T 65bd1232d19e4a2c903fa19ede36a9ba P 54fbce87fe064ab5a46db4b4197a093d: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 13:59:44.632259 24884 catalog_manager.cc:5484] T 68513fdc8f5745088711a271afdbfe68 P 54fbce87fe064ab5a46db4b4197a093d reported cstate change: term changed from 0 to 1, leader changed from <none> to 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65). New cstate: current_term: 1 leader_uuid: "54fbce87fe064ab5a46db4b4197a093d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } health_report { overall_health: UNKNOWN } } }
I20240502 13:59:44.632450 25355 raft_consensus.cc:348] T 758640af315340bd8d9e919888d30c86 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.632541 25355 raft_consensus.cc:374] T 758640af315340bd8d9e919888d30c86 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.632575 25355 raft_consensus.cc:729] T 758640af315340bd8d9e919888d30c86 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c7ba84b0bb4626af59a8c46dcbb9c1, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.632681 25355 consensus_queue.cc:260] T 758640af315340bd8d9e919888d30c86 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.632927 25355 ts_tablet_manager.cc:1398] T 758640af315340bd8d9e919888d30c86 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:44.633067 25354 tablet_bootstrap.cc:492] T 5617b9f3b6bb436e88a2aeaf051c4700 P f09f152a68f54c518c0c8e931fed5941: No bootstrap required, opened a new log
I20240502 13:59:44.633033 25356 raft_consensus.cc:348] T 65bd1232d19e4a2c903fa19ede36a9ba P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.633138 25356 raft_consensus.cc:374] T 65bd1232d19e4a2c903fa19ede36a9ba P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.633159 25354 ts_tablet_manager.cc:1367] T 5617b9f3b6bb436e88a2aeaf051c4700 P f09f152a68f54c518c0c8e931fed5941: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 13:59:44.633169 25356 raft_consensus.cc:729] T 65bd1232d19e4a2c903fa19ede36a9ba P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54fbce87fe064ab5a46db4b4197a093d, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.633303 25356 consensus_queue.cc:260] T 65bd1232d19e4a2c903fa19ede36a9ba P 54fbce87fe064ab5a46db4b4197a093d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.633563 25356 ts_tablet_manager.cc:1398] T 65bd1232d19e4a2c903fa19ede36a9ba P 54fbce87fe064ab5a46db4b4197a093d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:44.633594 25354 raft_consensus.cc:348] T 5617b9f3b6bb436e88a2aeaf051c4700 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.633674 25354 raft_consensus.cc:374] T 5617b9f3b6bb436e88a2aeaf051c4700 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.633704 25354 raft_consensus.cc:729] T 5617b9f3b6bb436e88a2aeaf051c4700 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f09f152a68f54c518c0c8e931fed5941, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.633723 25356 tablet_bootstrap.cc:492] T 758640af315340bd8d9e919888d30c86 P 54fbce87fe064ab5a46db4b4197a093d: Bootstrap starting.
I20240502 13:59:44.633818 25354 consensus_queue.cc:260] T 5617b9f3b6bb436e88a2aeaf051c4700 P f09f152a68f54c518c0c8e931fed5941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.634739 25412 raft_consensus.cc:482] T 5617b9f3b6bb436e88a2aeaf051c4700 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:44.634891 25412 raft_consensus.cc:504] T 5617b9f3b6bb436e88a2aeaf051c4700 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.634871 25354 ts_tablet_manager.cc:1398] T 5617b9f3b6bb436e88a2aeaf051c4700 P f09f152a68f54c518c0c8e931fed5941: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 13:59:44.634979 25356 tablet_bootstrap.cc:654] T 758640af315340bd8d9e919888d30c86 P 54fbce87fe064ab5a46db4b4197a093d: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.635244 25354 tablet_bootstrap.cc:492] T 34007af599ef4691977c361f01f22767 P f09f152a68f54c518c0c8e931fed5941: Bootstrap starting.
I20240502 13:59:44.635479 25412 leader_election.cc:288] T 5617b9f3b6bb436e88a2aeaf051c4700 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66:40803), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.635753 25286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5617b9f3b6bb436e88a2aeaf051c4700" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941" is_pre_election: true
I20240502 13:59:44.635861 25286 raft_consensus.cc:2454] T 5617b9f3b6bb436e88a2aeaf051c4700 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 0.
I20240502 13:59:44.636101 24962 leader_election.cc:302] T 5617b9f3b6bb436e88a2aeaf051c4700 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54fbce87fe064ab5a46db4b4197a093d, f09f152a68f54c518c0c8e931fed5941; no voters:
I20240502 13:59:44.636256 25412 raft_consensus.cc:2781] T 5617b9f3b6bb436e88a2aeaf051c4700 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:44.636307 25412 raft_consensus.cc:482] T 5617b9f3b6bb436e88a2aeaf051c4700 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:44.636317 25159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5617b9f3b6bb436e88a2aeaf051c4700" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" is_pre_election: true
I20240502 13:59:44.636384 25412 raft_consensus.cc:3037] T 5617b9f3b6bb436e88a2aeaf051c4700 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.636445 25159 raft_consensus.cc:2454] T 5617b9f3b6bb436e88a2aeaf051c4700 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 0.
I20240502 13:59:44.636857 25356 tablet_bootstrap.cc:492] T 758640af315340bd8d9e919888d30c86 P 54fbce87fe064ab5a46db4b4197a093d: No bootstrap required, opened a new log
I20240502 13:59:44.636930 25356 ts_tablet_manager.cc:1367] T 758640af315340bd8d9e919888d30c86 P 54fbce87fe064ab5a46db4b4197a093d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 13:59:44.637362 25412 raft_consensus.cc:504] T 5617b9f3b6bb436e88a2aeaf051c4700 P 54fbce87fe064ab5a46db4b4197a093d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.637593 25356 raft_consensus.cc:348] T 758640af315340bd8d9e919888d30c86 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.637682 25356 raft_consensus.cc:374] T 758640af315340bd8d9e919888d30c86 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.637720 25356 raft_consensus.cc:729] T 758640af315340bd8d9e919888d30c86 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54fbce87fe064ab5a46db4b4197a093d, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.637738 25412 leader_election.cc:288] T 5617b9f3b6bb436e88a2aeaf051c4700 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 election: Requested vote from peers 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66:40803), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.637830 25356 consensus_queue.cc:260] T 758640af315340bd8d9e919888d30c86 P 54fbce87fe064ab5a46db4b4197a093d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.637975 25159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5617b9f3b6bb436e88a2aeaf051c4700" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1"
I20240502 13:59:44.638067 25159 raft_consensus.cc:3037] T 5617b9f3b6bb436e88a2aeaf051c4700 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.638067 25286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5617b9f3b6bb436e88a2aeaf051c4700" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941"
I20240502 13:59:44.638162 25356 ts_tablet_manager.cc:1398] T 758640af315340bd8d9e919888d30c86 P 54fbce87fe064ab5a46db4b4197a093d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:44.638161 25286 raft_consensus.cc:3037] T 5617b9f3b6bb436e88a2aeaf051c4700 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.638330 25356 tablet_bootstrap.cc:492] T c8a7d300caa44b56918dc7045722e3c1 P 54fbce87fe064ab5a46db4b4197a093d: Bootstrap starting.
I20240502 13:59:44.638804 25159 raft_consensus.cc:2454] T 5617b9f3b6bb436e88a2aeaf051c4700 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 1.
I20240502 13:59:44.639037 25286 raft_consensus.cc:2454] T 5617b9f3b6bb436e88a2aeaf051c4700 P f09f152a68f54c518c0c8e931fed5941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 1.
I20240502 13:59:44.639086 24960 leader_election.cc:302] T 5617b9f3b6bb436e88a2aeaf051c4700 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45c7ba84b0bb4626af59a8c46dcbb9c1, 54fbce87fe064ab5a46db4b4197a093d; no voters:
I20240502 13:59:44.639233 25412 raft_consensus.cc:2781] T 5617b9f3b6bb436e88a2aeaf051c4700 P 54fbce87fe064ab5a46db4b4197a093d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:44.639319 25412 raft_consensus.cc:686] T 5617b9f3b6bb436e88a2aeaf051c4700 P 54fbce87fe064ab5a46db4b4197a093d [term 1 LEADER]: Becoming Leader. State: Replica: 54fbce87fe064ab5a46db4b4197a093d, State: Running, Role: LEADER
I20240502 13:59:44.639437 25412 consensus_queue.cc:237] T 5617b9f3b6bb436e88a2aeaf051c4700 P 54fbce87fe064ab5a46db4b4197a093d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.640141 25356 tablet_bootstrap.cc:654] T c8a7d300caa44b56918dc7045722e3c1 P 54fbce87fe064ab5a46db4b4197a093d: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.640141 25354 tablet_bootstrap.cc:654] T 34007af599ef4691977c361f01f22767 P f09f152a68f54c518c0c8e931fed5941: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.641019 24881 catalog_manager.cc:5484] T 5617b9f3b6bb436e88a2aeaf051c4700 P 54fbce87fe064ab5a46db4b4197a093d reported cstate change: term changed from 0 to 1, leader changed from <none> to 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65). New cstate: current_term: 1 leader_uuid: "54fbce87fe064ab5a46db4b4197a093d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } health_report { overall_health: UNKNOWN } } }
I20240502 13:59:44.641613 25356 tablet_bootstrap.cc:492] T c8a7d300caa44b56918dc7045722e3c1 P 54fbce87fe064ab5a46db4b4197a093d: No bootstrap required, opened a new log
I20240502 13:59:44.641695 25356 ts_tablet_manager.cc:1367] T c8a7d300caa44b56918dc7045722e3c1 P 54fbce87fe064ab5a46db4b4197a093d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 13:59:44.641777 25354 tablet_bootstrap.cc:492] T 34007af599ef4691977c361f01f22767 P f09f152a68f54c518c0c8e931fed5941: No bootstrap required, opened a new log
I20240502 13:59:44.641851 25354 ts_tablet_manager.cc:1367] T 34007af599ef4691977c361f01f22767 P f09f152a68f54c518c0c8e931fed5941: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20240502 13:59:44.642010 25356 raft_consensus.cc:348] T c8a7d300caa44b56918dc7045722e3c1 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.642077 25356 raft_consensus.cc:374] T c8a7d300caa44b56918dc7045722e3c1 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.642096 25356 raft_consensus.cc:729] T c8a7d300caa44b56918dc7045722e3c1 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54fbce87fe064ab5a46db4b4197a093d, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.642187 25356 consensus_queue.cc:260] T c8a7d300caa44b56918dc7045722e3c1 P 54fbce87fe064ab5a46db4b4197a093d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.642292 25354 raft_consensus.cc:348] T 34007af599ef4691977c361f01f22767 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.642400 25354 raft_consensus.cc:374] T 34007af599ef4691977c361f01f22767 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.642423 25356 ts_tablet_manager.cc:1398] T c8a7d300caa44b56918dc7045722e3c1 P 54fbce87fe064ab5a46db4b4197a093d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:44.642452 25354 raft_consensus.cc:729] T 34007af599ef4691977c361f01f22767 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f09f152a68f54c518c0c8e931fed5941, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.642571 25354 consensus_queue.cc:260] T 34007af599ef4691977c361f01f22767 P f09f152a68f54c518c0c8e931fed5941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.642993 25354 ts_tablet_manager.cc:1398] T 34007af599ef4691977c361f01f22767 P f09f152a68f54c518c0c8e931fed5941: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 13:59:44.642728 25356 tablet_bootstrap.cc:492] T 34007af599ef4691977c361f01f22767 P 54fbce87fe064ab5a46db4b4197a093d: Bootstrap starting.
I20240502 13:59:44.644256 25356 tablet_bootstrap.cc:654] T 34007af599ef4691977c361f01f22767 P 54fbce87fe064ab5a46db4b4197a093d: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:44.645351 25356 tablet_bootstrap.cc:492] T 34007af599ef4691977c361f01f22767 P 54fbce87fe064ab5a46db4b4197a093d: No bootstrap required, opened a new log
I20240502 13:59:44.645427 25356 ts_tablet_manager.cc:1367] T 34007af599ef4691977c361f01f22767 P 54fbce87fe064ab5a46db4b4197a093d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 13:59:44.645860 25356 raft_consensus.cc:348] T 34007af599ef4691977c361f01f22767 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.645941 25356 raft_consensus.cc:374] T 34007af599ef4691977c361f01f22767 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:44.645973 25356 raft_consensus.cc:729] T 34007af599ef4691977c361f01f22767 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54fbce87fe064ab5a46db4b4197a093d, State: Initialized, Role: FOLLOWER
I20240502 13:59:44.646090 25356 consensus_queue.cc:260] T 34007af599ef4691977c361f01f22767 P 54fbce87fe064ab5a46db4b4197a093d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.646297 25356 ts_tablet_manager.cc:1398] T 34007af599ef4691977c361f01f22767 P 54fbce87fe064ab5a46db4b4197a093d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:44.648916 25366 raft_consensus.cc:482] T 34007af599ef4691977c361f01f22767 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:44.648996 25366 raft_consensus.cc:504] T 34007af599ef4691977c361f01f22767 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.649926 25366 leader_election.cc:288] T 34007af599ef4691977c361f01f22767 P f09f152a68f54c518c0c8e931fed5941 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65:34575), 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66:40803)
I20240502 13:59:44.654084 25026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34007af599ef4691977c361f01f22767" candidate_uuid: "f09f152a68f54c518c0c8e931fed5941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54fbce87fe064ab5a46db4b4197a093d" is_pre_election: true
I20240502 13:59:44.654199 25026 raft_consensus.cc:2454] T 34007af599ef4691977c361f01f22767 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f09f152a68f54c518c0c8e931fed5941 in term 0.
I20240502 13:59:44.654230 25159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34007af599ef4691977c361f01f22767" candidate_uuid: "f09f152a68f54c518c0c8e931fed5941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" is_pre_election: true
I20240502 13:59:44.654369 25159 raft_consensus.cc:2454] T 34007af599ef4691977c361f01f22767 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f09f152a68f54c518c0c8e931fed5941 in term 0.
I20240502 13:59:44.654505 25225 leader_election.cc:302] T 34007af599ef4691977c361f01f22767 P f09f152a68f54c518c0c8e931fed5941 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54fbce87fe064ab5a46db4b4197a093d, f09f152a68f54c518c0c8e931fed5941; no voters:
I20240502 13:59:44.654762 25366 raft_consensus.cc:2781] T 34007af599ef4691977c361f01f22767 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:44.654821 25366 raft_consensus.cc:482] T 34007af599ef4691977c361f01f22767 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:44.654876 25366 raft_consensus.cc:3037] T 34007af599ef4691977c361f01f22767 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.655648 25366 raft_consensus.cc:504] T 34007af599ef4691977c361f01f22767 P f09f152a68f54c518c0c8e931fed5941 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.655937 25366 leader_election.cc:288] T 34007af599ef4691977c361f01f22767 P f09f152a68f54c518c0c8e931fed5941 [CANDIDATE]: Term 1 election: Requested vote from peers 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65:34575), 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66:40803)
I20240502 13:59:44.656236 25026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34007af599ef4691977c361f01f22767" candidate_uuid: "f09f152a68f54c518c0c8e931fed5941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54fbce87fe064ab5a46db4b4197a093d"
I20240502 13:59:44.656256 25159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34007af599ef4691977c361f01f22767" candidate_uuid: "f09f152a68f54c518c0c8e931fed5941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1"
I20240502 13:59:44.656380 25159 raft_consensus.cc:3037] T 34007af599ef4691977c361f01f22767 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.656378 25026 raft_consensus.cc:3037] T 34007af599ef4691977c361f01f22767 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.657042 25026 raft_consensus.cc:2454] T 34007af599ef4691977c361f01f22767 P 54fbce87fe064ab5a46db4b4197a093d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f09f152a68f54c518c0c8e931fed5941 in term 1.
I20240502 13:59:44.657042 25159 raft_consensus.cc:2454] T 34007af599ef4691977c361f01f22767 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f09f152a68f54c518c0c8e931fed5941 in term 1.
I20240502 13:59:44.657296 25223 leader_election.cc:302] T 34007af599ef4691977c361f01f22767 P f09f152a68f54c518c0c8e931fed5941 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45c7ba84b0bb4626af59a8c46dcbb9c1, f09f152a68f54c518c0c8e931fed5941; no voters:
I20240502 13:59:44.657467 25366 raft_consensus.cc:2781] T 34007af599ef4691977c361f01f22767 P f09f152a68f54c518c0c8e931fed5941 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:44.657554 25366 raft_consensus.cc:686] T 34007af599ef4691977c361f01f22767 P f09f152a68f54c518c0c8e931fed5941 [term 1 LEADER]: Becoming Leader. State: Replica: f09f152a68f54c518c0c8e931fed5941, State: Running, Role: LEADER
I20240502 13:59:44.657856 25366 consensus_queue.cc:237] T 34007af599ef4691977c361f01f22767 P f09f152a68f54c518c0c8e931fed5941 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.660641 24884 catalog_manager.cc:5484] T 34007af599ef4691977c361f01f22767 P f09f152a68f54c518c0c8e931fed5941 reported cstate change: term changed from 0 to 1, leader changed from <none> to f09f152a68f54c518c0c8e931fed5941 (127.24.66.67). New cstate: current_term: 1 leader_uuid: "f09f152a68f54c518c0c8e931fed5941" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } health_report { overall_health: HEALTHY } } }
I20240502 13:59:44.670310 25404 raft_consensus.cc:482] T 2b5e884f76de4e749872a7b9a2bd8493 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:44.670423 25404 raft_consensus.cc:504] T 2b5e884f76de4e749872a7b9a2bd8493 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.670580 25412 raft_consensus.cc:482] T 65bd1232d19e4a2c903fa19ede36a9ba P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:44.670694 25412 raft_consensus.cc:504] T 65bd1232d19e4a2c903fa19ede36a9ba P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.670886 25404 leader_election.cc:288] T 2b5e884f76de4e749872a7b9a2bd8493 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65:34575), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.671094 25412 leader_election.cc:288] T 65bd1232d19e4a2c903fa19ede36a9ba P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66:40803), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.671293 25026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b5e884f76de4e749872a7b9a2bd8493" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54fbce87fe064ab5a46db4b4197a093d" is_pre_election: true
I20240502 13:59:44.671305 25286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b5e884f76de4e749872a7b9a2bd8493" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941" is_pre_election: true
I20240502 13:59:44.671420 25286 raft_consensus.cc:2454] T 2b5e884f76de4e749872a7b9a2bd8493 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 0.
I20240502 13:59:44.671420 25026 raft_consensus.cc:2454] T 2b5e884f76de4e749872a7b9a2bd8493 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 0.
I20240502 13:59:44.671481 25159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65bd1232d19e4a2c903fa19ede36a9ba" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" is_pre_election: true
I20240502 13:59:44.671483 25285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65bd1232d19e4a2c903fa19ede36a9ba" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941" is_pre_election: true
I20240502 13:59:44.671592 25285 raft_consensus.cc:2454] T 65bd1232d19e4a2c903fa19ede36a9ba P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 0.
I20240502 13:59:44.671592 25159 raft_consensus.cc:2454] T 65bd1232d19e4a2c903fa19ede36a9ba P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 0.
I20240502 13:59:44.671687 25094 leader_election.cc:302] T 2b5e884f76de4e749872a7b9a2bd8493 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45c7ba84b0bb4626af59a8c46dcbb9c1, 54fbce87fe064ab5a46db4b4197a093d; no voters:
I20240502 13:59:44.671882 24960 leader_election.cc:302] T 65bd1232d19e4a2c903fa19ede36a9ba P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45c7ba84b0bb4626af59a8c46dcbb9c1, 54fbce87fe064ab5a46db4b4197a093d; no voters:
I20240502 13:59:44.671911 25404 raft_consensus.cc:2781] T 2b5e884f76de4e749872a7b9a2bd8493 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:44.671957 25404 raft_consensus.cc:482] T 2b5e884f76de4e749872a7b9a2bd8493 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:44.671996 25404 raft_consensus.cc:3037] T 2b5e884f76de4e749872a7b9a2bd8493 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.672039 25412 raft_consensus.cc:2781] T 65bd1232d19e4a2c903fa19ede36a9ba P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:44.672097 25412 raft_consensus.cc:482] T 65bd1232d19e4a2c903fa19ede36a9ba P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:44.672142 25412 raft_consensus.cc:3037] T 65bd1232d19e4a2c903fa19ede36a9ba P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.673095 25412 raft_consensus.cc:504] T 65bd1232d19e4a2c903fa19ede36a9ba P 54fbce87fe064ab5a46db4b4197a093d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.673095 25404 raft_consensus.cc:504] T 2b5e884f76de4e749872a7b9a2bd8493 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.673435 25404 leader_election.cc:288] T 2b5e884f76de4e749872a7b9a2bd8493 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 election: Requested vote from peers 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65:34575), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.673455 25412 leader_election.cc:288] T 65bd1232d19e4a2c903fa19ede36a9ba P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 election: Requested vote from peers 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66:40803), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.673746 25026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b5e884f76de4e749872a7b9a2bd8493" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54fbce87fe064ab5a46db4b4197a093d"
I20240502 13:59:44.673748 25159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65bd1232d19e4a2c903fa19ede36a9ba" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1"
I20240502 13:59:44.673842 25026 raft_consensus.cc:3037] T 2b5e884f76de4e749872a7b9a2bd8493 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.673847 25159 raft_consensus.cc:3037] T 65bd1232d19e4a2c903fa19ede36a9ba P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.673833 25285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b5e884f76de4e749872a7b9a2bd8493" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941"
I20240502 13:59:44.673856 25286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65bd1232d19e4a2c903fa19ede36a9ba" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941"
I20240502 13:59:44.673928 25285 raft_consensus.cc:3037] T 2b5e884f76de4e749872a7b9a2bd8493 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.673949 25286 raft_consensus.cc:3037] T 65bd1232d19e4a2c903fa19ede36a9ba P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.674700 25026 raft_consensus.cc:2454] T 2b5e884f76de4e749872a7b9a2bd8493 P 54fbce87fe064ab5a46db4b4197a093d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 1.
I20240502 13:59:44.674785 25159 raft_consensus.cc:2454] T 65bd1232d19e4a2c903fa19ede36a9ba P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 1.
I20240502 13:59:44.674801 25285 raft_consensus.cc:2454] T 2b5e884f76de4e749872a7b9a2bd8493 P f09f152a68f54c518c0c8e931fed5941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 1.
I20240502 13:59:44.674876 25286 raft_consensus.cc:2454] T 65bd1232d19e4a2c903fa19ede36a9ba P f09f152a68f54c518c0c8e931fed5941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 1.
I20240502 13:59:44.675030 24960 leader_election.cc:302] T 65bd1232d19e4a2c903fa19ede36a9ba P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45c7ba84b0bb4626af59a8c46dcbb9c1, 54fbce87fe064ab5a46db4b4197a093d; no voters:
I20240502 13:59:44.675207 25412 raft_consensus.cc:2781] T 65bd1232d19e4a2c903fa19ede36a9ba P 54fbce87fe064ab5a46db4b4197a093d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:44.675218 25094 leader_election.cc:302] T 2b5e884f76de4e749872a7b9a2bd8493 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45c7ba84b0bb4626af59a8c46dcbb9c1, f09f152a68f54c518c0c8e931fed5941; no voters:
I20240502 13:59:44.675310 25412 raft_consensus.cc:686] T 65bd1232d19e4a2c903fa19ede36a9ba P 54fbce87fe064ab5a46db4b4197a093d [term 1 LEADER]: Becoming Leader. State: Replica: 54fbce87fe064ab5a46db4b4197a093d, State: Running, Role: LEADER
I20240502 13:59:44.675374 25404 raft_consensus.cc:2781] T 2b5e884f76de4e749872a7b9a2bd8493 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:44.675453 25404 raft_consensus.cc:686] T 2b5e884f76de4e749872a7b9a2bd8493 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 LEADER]: Becoming Leader. State: Replica: 45c7ba84b0bb4626af59a8c46dcbb9c1, State: Running, Role: LEADER
I20240502 13:59:44.675443 25412 consensus_queue.cc:237] T 65bd1232d19e4a2c903fa19ede36a9ba P 54fbce87fe064ab5a46db4b4197a093d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.675561 25404 consensus_queue.cc:237] T 2b5e884f76de4e749872a7b9a2bd8493 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.677208 24881 catalog_manager.cc:5484] T 2b5e884f76de4e749872a7b9a2bd8493 P 45c7ba84b0bb4626af59a8c46dcbb9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66). New cstate: current_term: 1 leader_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } health_report { overall_health: UNKNOWN } } }
I20240502 13:59:44.677335 24884 catalog_manager.cc:5484] T 65bd1232d19e4a2c903fa19ede36a9ba P 54fbce87fe064ab5a46db4b4197a093d reported cstate change: term changed from 0 to 1, leader changed from <none> to 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65). New cstate: current_term: 1 leader_uuid: "54fbce87fe064ab5a46db4b4197a093d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } health_report { overall_health: UNKNOWN } } }
I20240502 13:59:44.682633 25404 raft_consensus.cc:482] T 758640af315340bd8d9e919888d30c86 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:44.682718 25404 raft_consensus.cc:504] T 758640af315340bd8d9e919888d30c86 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.682968 25404 leader_election.cc:288] T 758640af315340bd8d9e919888d30c86 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65:34575), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.683254 25026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "758640af315340bd8d9e919888d30c86" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54fbce87fe064ab5a46db4b4197a093d" is_pre_election: true
I20240502 13:59:44.683352 25026 raft_consensus.cc:2454] T 758640af315340bd8d9e919888d30c86 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 0.
I20240502 13:59:44.683344 25286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "758640af315340bd8d9e919888d30c86" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941" is_pre_election: true
I20240502 13:59:44.683465 25286 raft_consensus.cc:2454] T 758640af315340bd8d9e919888d30c86 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 0.
I20240502 13:59:44.683596 25094 leader_election.cc:302] T 758640af315340bd8d9e919888d30c86 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45c7ba84b0bb4626af59a8c46dcbb9c1, 54fbce87fe064ab5a46db4b4197a093d; no voters:
I20240502 13:59:44.683746 25404 raft_consensus.cc:2781] T 758640af315340bd8d9e919888d30c86 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:44.683808 25404 raft_consensus.cc:482] T 758640af315340bd8d9e919888d30c86 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:44.683835 25404 raft_consensus.cc:3037] T 758640af315340bd8d9e919888d30c86 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.684581 25404 raft_consensus.cc:504] T 758640af315340bd8d9e919888d30c86 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.684849 25404 leader_election.cc:288] T 758640af315340bd8d9e919888d30c86 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 election: Requested vote from peers 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65:34575), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.685124 25026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "758640af315340bd8d9e919888d30c86" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54fbce87fe064ab5a46db4b4197a093d"
I20240502 13:59:44.685230 25026 raft_consensus.cc:3037] T 758640af315340bd8d9e919888d30c86 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.685226 25286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "758640af315340bd8d9e919888d30c86" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941"
I20240502 13:59:44.685334 25286 raft_consensus.cc:3037] T 758640af315340bd8d9e919888d30c86 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.685907 25026 raft_consensus.cc:2454] T 758640af315340bd8d9e919888d30c86 P 54fbce87fe064ab5a46db4b4197a093d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 1.
I20240502 13:59:44.685978 25286 raft_consensus.cc:2454] T 758640af315340bd8d9e919888d30c86 P f09f152a68f54c518c0c8e931fed5941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 1.
I20240502 13:59:44.686139 25094 leader_election.cc:302] T 758640af315340bd8d9e919888d30c86 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45c7ba84b0bb4626af59a8c46dcbb9c1, 54fbce87fe064ab5a46db4b4197a093d; no voters:
I20240502 13:59:44.686291 25404 raft_consensus.cc:2781] T 758640af315340bd8d9e919888d30c86 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:44.686371 25404 raft_consensus.cc:686] T 758640af315340bd8d9e919888d30c86 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 LEADER]: Becoming Leader. State: Replica: 45c7ba84b0bb4626af59a8c46dcbb9c1, State: Running, Role: LEADER
I20240502 13:59:44.686498 25404 consensus_queue.cc:237] T 758640af315340bd8d9e919888d30c86 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.687934 24884 catalog_manager.cc:5484] T 758640af315340bd8d9e919888d30c86 P 45c7ba84b0bb4626af59a8c46dcbb9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66). New cstate: current_term: 1 leader_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } health_report { overall_health: UNKNOWN } } }
W20240502 13:59:44.695438 25337 tablet.cc:2369] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 13:59:44.698468 25412 raft_consensus.cc:482] T c8a7d300caa44b56918dc7045722e3c1 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:44.698565 25412 raft_consensus.cc:504] T c8a7d300caa44b56918dc7045722e3c1 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.698917 25412 leader_election.cc:288] T c8a7d300caa44b56918dc7045722e3c1 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66:40803), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.699244 25286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a7d300caa44b56918dc7045722e3c1" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941" is_pre_election: true
I20240502 13:59:44.699257 25159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a7d300caa44b56918dc7045722e3c1" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" is_pre_election: true
I20240502 13:59:44.699342 25286 raft_consensus.cc:2454] T c8a7d300caa44b56918dc7045722e3c1 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 0.
I20240502 13:59:44.699359 25159 raft_consensus.cc:2454] T c8a7d300caa44b56918dc7045722e3c1 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 0.
I20240502 13:59:44.699594 24962 leader_election.cc:302] T c8a7d300caa44b56918dc7045722e3c1 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54fbce87fe064ab5a46db4b4197a093d, f09f152a68f54c518c0c8e931fed5941; no voters:
I20240502 13:59:44.699754 25412 raft_consensus.cc:2781] T c8a7d300caa44b56918dc7045722e3c1 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:44.699817 25412 raft_consensus.cc:482] T c8a7d300caa44b56918dc7045722e3c1 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:44.699862 25412 raft_consensus.cc:3037] T c8a7d300caa44b56918dc7045722e3c1 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.700692 25412 raft_consensus.cc:504] T c8a7d300caa44b56918dc7045722e3c1 P 54fbce87fe064ab5a46db4b4197a093d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.700984 25412 leader_election.cc:288] T c8a7d300caa44b56918dc7045722e3c1 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 election: Requested vote from peers 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66:40803), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.701318 25286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a7d300caa44b56918dc7045722e3c1" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941"
I20240502 13:59:44.701409 25286 raft_consensus.cc:3037] T c8a7d300caa44b56918dc7045722e3c1 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.702055 25286 raft_consensus.cc:2454] T c8a7d300caa44b56918dc7045722e3c1 P f09f152a68f54c518c0c8e931fed5941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 1.
I20240502 13:59:44.702253 25159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8a7d300caa44b56918dc7045722e3c1" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1"
I20240502 13:59:44.702304 24962 leader_election.cc:302] T c8a7d300caa44b56918dc7045722e3c1 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54fbce87fe064ab5a46db4b4197a093d, f09f152a68f54c518c0c8e931fed5941; no voters:
I20240502 13:59:44.702363 25159 raft_consensus.cc:3037] T c8a7d300caa44b56918dc7045722e3c1 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.702445 25412 raft_consensus.cc:2781] T c8a7d300caa44b56918dc7045722e3c1 P 54fbce87fe064ab5a46db4b4197a093d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:44.702525 25412 raft_consensus.cc:686] T c8a7d300caa44b56918dc7045722e3c1 P 54fbce87fe064ab5a46db4b4197a093d [term 1 LEADER]: Becoming Leader. State: Replica: 54fbce87fe064ab5a46db4b4197a093d, State: Running, Role: LEADER
I20240502 13:59:44.702641 25412 consensus_queue.cc:237] T c8a7d300caa44b56918dc7045722e3c1 P 54fbce87fe064ab5a46db4b4197a093d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.703068 25159 raft_consensus.cc:2454] T c8a7d300caa44b56918dc7045722e3c1 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 1.
I20240502 13:59:44.704273 24884 catalog_manager.cc:5484] T c8a7d300caa44b56918dc7045722e3c1 P 54fbce87fe064ab5a46db4b4197a093d reported cstate change: term changed from 0 to 1, leader changed from <none> to 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65). New cstate: current_term: 1 leader_uuid: "54fbce87fe064ab5a46db4b4197a093d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } health_report { overall_health: UNKNOWN } } }
I20240502 13:59:44.721204 25366 raft_consensus.cc:482] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:44.721334 25408 raft_consensus.cc:482] T 0bd322784404440983b3ca1026f13c10 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:44.721351 25366 raft_consensus.cc:504] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.721427 25408 raft_consensus.cc:504] T 0bd322784404440983b3ca1026f13c10 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.721787 25366 leader_election.cc:288] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65:34575), 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66:40803)
I20240502 13:59:44.721797 25408 leader_election.cc:288] T 0bd322784404440983b3ca1026f13c10 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66:40803), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.722189 25156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bd322784404440983b3ca1026f13c10" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" is_pre_election: true
I20240502 13:59:44.722239 25286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bd322784404440983b3ca1026f13c10" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941" is_pre_election: true
I20240502 13:59:44.722235 25026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bd322784404440983b3ca1026f13c10" candidate_uuid: "f09f152a68f54c518c0c8e931fed5941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54fbce87fe064ab5a46db4b4197a093d" is_pre_election: true
I20240502 13:59:44.722183 25159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bd322784404440983b3ca1026f13c10" candidate_uuid: "f09f152a68f54c518c0c8e931fed5941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" is_pre_election: true
I20240502 13:59:44.722352 25286 raft_consensus.cc:2454] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 0.
I20240502 13:59:44.722352 25026 raft_consensus.cc:2454] T 0bd322784404440983b3ca1026f13c10 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f09f152a68f54c518c0c8e931fed5941 in term 0.
I20240502 13:59:44.722362 25156 raft_consensus.cc:2454] T 0bd322784404440983b3ca1026f13c10 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 0.
I20240502 13:59:44.722633 24960 leader_election.cc:302] T 0bd322784404440983b3ca1026f13c10 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45c7ba84b0bb4626af59a8c46dcbb9c1, 54fbce87fe064ab5a46db4b4197a093d; no voters:
I20240502 13:59:44.722673 25225 leader_election.cc:302] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54fbce87fe064ab5a46db4b4197a093d, f09f152a68f54c518c0c8e931fed5941; no voters:
I20240502 13:59:44.722797 25408 raft_consensus.cc:2781] T 0bd322784404440983b3ca1026f13c10 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:44.722860 25408 raft_consensus.cc:482] T 0bd322784404440983b3ca1026f13c10 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:44.722906 25408 raft_consensus.cc:3037] T 0bd322784404440983b3ca1026f13c10 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.722923 25366 raft_consensus.cc:2781] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:44.723002 25366 raft_consensus.cc:482] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:44.723052 25366 raft_consensus.cc:3037] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.723719 25408 raft_consensus.cc:504] T 0bd322784404440983b3ca1026f13c10 P 54fbce87fe064ab5a46db4b4197a093d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.723762 25366 raft_consensus.cc:504] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.724025 25408 leader_election.cc:288] T 0bd322784404440983b3ca1026f13c10 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 election: Requested vote from peers 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66:40803), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.724056 25366 leader_election.cc:288] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941 [CANDIDATE]: Term 1 election: Requested vote from peers 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65:34575), 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66:40803)
I20240502 13:59:44.724315 25026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bd322784404440983b3ca1026f13c10" candidate_uuid: "f09f152a68f54c518c0c8e931fed5941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54fbce87fe064ab5a46db4b4197a093d"
I20240502 13:59:44.724402 25286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bd322784404440983b3ca1026f13c10" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941"
I20240502 13:59:44.724382 25156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bd322784404440983b3ca1026f13c10" candidate_uuid: "f09f152a68f54c518c0c8e931fed5941" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1"
I20240502 13:59:44.724495 25156 raft_consensus.cc:3037] T 0bd322784404440983b3ca1026f13c10 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.724527 25026 raft_consensus.cc:2379] T 0bd322784404440983b3ca1026f13c10 P 54fbce87fe064ab5a46db4b4197a093d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f09f152a68f54c518c0c8e931fed5941 in current term 1: Already voted for candidate 54fbce87fe064ab5a46db4b4197a093d in this term.
I20240502 13:59:44.724535 25286 raft_consensus.cc:2379] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 54fbce87fe064ab5a46db4b4197a093d in current term 1: Already voted for candidate f09f152a68f54c518c0c8e931fed5941 in this term.
I20240502 13:59:44.724391 25159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bd322784404440983b3ca1026f13c10" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1"
I20240502 13:59:44.725294 24960 leader_election.cc:302] T 0bd322784404440983b3ca1026f13c10 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54fbce87fe064ab5a46db4b4197a093d; no voters: 45c7ba84b0bb4626af59a8c46dcbb9c1, f09f152a68f54c518c0c8e931fed5941
I20240502 13:59:44.725421 25156 raft_consensus.cc:2454] T 0bd322784404440983b3ca1026f13c10 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f09f152a68f54c518c0c8e931fed5941 in term 1.
I20240502 13:59:44.725432 25408 raft_consensus.cc:2726] T 0bd322784404440983b3ca1026f13c10 P 54fbce87fe064ab5a46db4b4197a093d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240502 13:59:44.725682 25223 leader_election.cc:302] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45c7ba84b0bb4626af59a8c46dcbb9c1, f09f152a68f54c518c0c8e931fed5941; no voters: 54fbce87fe064ab5a46db4b4197a093d
I20240502 13:59:44.725859 25366 raft_consensus.cc:2781] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:44.725941 25366 raft_consensus.cc:686] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941 [term 1 LEADER]: Becoming Leader. State: Replica: f09f152a68f54c518c0c8e931fed5941, State: Running, Role: LEADER
I20240502 13:59:44.726089 25366 consensus_queue.cc:237] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.727636 24884 catalog_manager.cc:5484] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941 reported cstate change: term changed from 0 to 1, leader changed from <none> to f09f152a68f54c518c0c8e931fed5941 (127.24.66.67). New cstate: current_term: 1 leader_uuid: "f09f152a68f54c518c0c8e931fed5941" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } health_report { overall_health: HEALTHY } } }
W20240502 13:59:44.779804 25206 tablet.cc:2369] T 0bd322784404440983b3ca1026f13c10 P 45c7ba84b0bb4626af59a8c46dcbb9c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 13:59:44.789275 25404 raft_consensus.cc:482] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:44.789413 25404 raft_consensus.cc:504] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.789837 25404 leader_election.cc:288] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65:34575), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.790405 25026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b30b6eada4345e8a8bc4e7bda88c8d5" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54fbce87fe064ab5a46db4b4197a093d" is_pre_election: true
I20240502 13:59:44.790405 25286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b30b6eada4345e8a8bc4e7bda88c8d5" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941" is_pre_election: true
I20240502 13:59:44.790560 25026 raft_consensus.cc:2454] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 0.
I20240502 13:59:44.790557 25286 raft_consensus.cc:2454] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 0.
I20240502 13:59:44.790854 25094 leader_election.cc:302] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45c7ba84b0bb4626af59a8c46dcbb9c1, 54fbce87fe064ab5a46db4b4197a093d; no voters:
I20240502 13:59:44.791013 25404 raft_consensus.cc:2781] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:44.791069 25404 raft_consensus.cc:482] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:44.791116 25404 raft_consensus.cc:3037] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.791884 25404 raft_consensus.cc:504] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.792192 25404 leader_election.cc:288] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 election: Requested vote from peers 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65:34575), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.792619 25286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b30b6eada4345e8a8bc4e7bda88c8d5" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941"
I20240502 13:59:44.792619 25026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b30b6eada4345e8a8bc4e7bda88c8d5" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54fbce87fe064ab5a46db4b4197a093d"
I20240502 13:59:44.792742 25026 raft_consensus.cc:3037] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.792742 25286 raft_consensus.cc:3037] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.793666 25026 raft_consensus.cc:2454] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 54fbce87fe064ab5a46db4b4197a093d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 1.
I20240502 13:59:44.793676 25286 raft_consensus.cc:2454] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P f09f152a68f54c518c0c8e931fed5941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 1.
I20240502 13:59:44.793967 25094 leader_election.cc:302] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45c7ba84b0bb4626af59a8c46dcbb9c1, f09f152a68f54c518c0c8e931fed5941; no voters:
I20240502 13:59:44.794133 25404 raft_consensus.cc:2781] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:44.794214 25404 raft_consensus.cc:686] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 LEADER]: Becoming Leader. State: Replica: 45c7ba84b0bb4626af59a8c46dcbb9c1, State: Running, Role: LEADER
I20240502 13:59:44.794358 25404 consensus_queue.cc:237] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.796057 24884 catalog_manager.cc:5484] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 45c7ba84b0bb4626af59a8c46dcbb9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66). New cstate: current_term: 1 leader_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } health_report { overall_health: UNKNOWN } } }
I20240502 13:59:44.958981 25404 raft_consensus.cc:482] T adc3311b364245c69107c0cef3c5ce8a P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:44.959116 25404 raft_consensus.cc:504] T adc3311b364245c69107c0cef3c5ce8a P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.959486 25404 leader_election.cc:288] T adc3311b364245c69107c0cef3c5ce8a P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65:34575), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.960060 25286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adc3311b364245c69107c0cef3c5ce8a" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941" is_pre_election: true
I20240502 13:59:44.960089 25026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adc3311b364245c69107c0cef3c5ce8a" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54fbce87fe064ab5a46db4b4197a093d" is_pre_election: true
I20240502 13:59:44.960186 25286 raft_consensus.cc:2454] T adc3311b364245c69107c0cef3c5ce8a P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 0.
I20240502 13:59:44.960258 25026 raft_consensus.cc:2454] T adc3311b364245c69107c0cef3c5ce8a P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 0.
I20240502 13:59:44.960526 25094 leader_election.cc:302] T adc3311b364245c69107c0cef3c5ce8a P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45c7ba84b0bb4626af59a8c46dcbb9c1, f09f152a68f54c518c0c8e931fed5941; no voters:
I20240502 13:59:44.960700 25404 raft_consensus.cc:2781] T adc3311b364245c69107c0cef3c5ce8a P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:44.960755 25404 raft_consensus.cc:482] T adc3311b364245c69107c0cef3c5ce8a P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:44.960815 25404 raft_consensus.cc:3037] T adc3311b364245c69107c0cef3c5ce8a P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.961656 25404 raft_consensus.cc:504] T adc3311b364245c69107c0cef3c5ce8a P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.961995 25404 leader_election.cc:288] T adc3311b364245c69107c0cef3c5ce8a P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 election: Requested vote from peers 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65:34575), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.962359 25026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adc3311b364245c69107c0cef3c5ce8a" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54fbce87fe064ab5a46db4b4197a093d"
I20240502 13:59:44.962385 25286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adc3311b364245c69107c0cef3c5ce8a" candidate_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941"
I20240502 13:59:44.962469 25026 raft_consensus.cc:3037] T adc3311b364245c69107c0cef3c5ce8a P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.962510 25286 raft_consensus.cc:3037] T adc3311b364245c69107c0cef3c5ce8a P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.963265 25026 raft_consensus.cc:2454] T adc3311b364245c69107c0cef3c5ce8a P 54fbce87fe064ab5a46db4b4197a093d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 1.
I20240502 13:59:44.963287 25286 raft_consensus.cc:2454] T adc3311b364245c69107c0cef3c5ce8a P f09f152a68f54c518c0c8e931fed5941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45c7ba84b0bb4626af59a8c46dcbb9c1 in term 1.
I20240502 13:59:44.963572 25094 leader_election.cc:302] T adc3311b364245c69107c0cef3c5ce8a P 45c7ba84b0bb4626af59a8c46dcbb9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45c7ba84b0bb4626af59a8c46dcbb9c1, 54fbce87fe064ab5a46db4b4197a093d; no voters:
I20240502 13:59:44.963738 25404 raft_consensus.cc:2781] T adc3311b364245c69107c0cef3c5ce8a P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:44.963831 25404 raft_consensus.cc:686] T adc3311b364245c69107c0cef3c5ce8a P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 LEADER]: Becoming Leader. State: Replica: 45c7ba84b0bb4626af59a8c46dcbb9c1, State: Running, Role: LEADER
I20240502 13:59:44.963979 25404 consensus_queue.cc:237] T adc3311b364245c69107c0cef3c5ce8a P 45c7ba84b0bb4626af59a8c46dcbb9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.965700 24884 catalog_manager.cc:5484] T adc3311b364245c69107c0cef3c5ce8a P 45c7ba84b0bb4626af59a8c46dcbb9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66). New cstate: current_term: 1 leader_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } health_report { overall_health: UNKNOWN } } }
I20240502 13:59:44.976759 25408 raft_consensus.cc:482] T 241895b448ae436b9b527a94c3331468 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:44.976900 25408 raft_consensus.cc:504] T 241895b448ae436b9b527a94c3331468 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.977396 25408 leader_election.cc:288] T 241895b448ae436b9b527a94c3331468 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66:40803), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.977880 25286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241895b448ae436b9b527a94c3331468" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941" is_pre_election: true
I20240502 13:59:44.977919 25156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241895b448ae436b9b527a94c3331468" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" is_pre_election: true
I20240502 13:59:44.978025 25286 raft_consensus.cc:2454] T 241895b448ae436b9b527a94c3331468 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 0.
I20240502 13:59:44.978057 25156 raft_consensus.cc:2454] T 241895b448ae436b9b527a94c3331468 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 0.
I20240502 13:59:44.978292 24962 leader_election.cc:302] T 241895b448ae436b9b527a94c3331468 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54fbce87fe064ab5a46db4b4197a093d, f09f152a68f54c518c0c8e931fed5941; no voters:
I20240502 13:59:44.978464 25408 raft_consensus.cc:2781] T 241895b448ae436b9b527a94c3331468 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:44.978523 25408 raft_consensus.cc:482] T 241895b448ae436b9b527a94c3331468 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:44.978576 25408 raft_consensus.cc:3037] T 241895b448ae436b9b527a94c3331468 P 54fbce87fe064ab5a46db4b4197a093d [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.979370 25408 raft_consensus.cc:504] T 241895b448ae436b9b527a94c3331468 P 54fbce87fe064ab5a46db4b4197a093d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.979682 25408 leader_election.cc:288] T 241895b448ae436b9b527a94c3331468 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 election: Requested vote from peers 45c7ba84b0bb4626af59a8c46dcbb9c1 (127.24.66.66:40803), f09f152a68f54c518c0c8e931fed5941 (127.24.66.67:39311)
I20240502 13:59:44.979982 25156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241895b448ae436b9b527a94c3331468" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1"
I20240502 13:59:44.980038 25286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241895b448ae436b9b527a94c3331468" candidate_uuid: "54fbce87fe064ab5a46db4b4197a093d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f09f152a68f54c518c0c8e931fed5941"
I20240502 13:59:44.980101 25156 raft_consensus.cc:3037] T 241895b448ae436b9b527a94c3331468 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.980134 25286 raft_consensus.cc:3037] T 241895b448ae436b9b527a94c3331468 P f09f152a68f54c518c0c8e931fed5941 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:44.980870 25286 raft_consensus.cc:2454] T 241895b448ae436b9b527a94c3331468 P f09f152a68f54c518c0c8e931fed5941 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 1.
I20240502 13:59:44.980886 25156 raft_consensus.cc:2454] T 241895b448ae436b9b527a94c3331468 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54fbce87fe064ab5a46db4b4197a093d in term 1.
I20240502 13:59:44.981132 24962 leader_election.cc:302] T 241895b448ae436b9b527a94c3331468 P 54fbce87fe064ab5a46db4b4197a093d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54fbce87fe064ab5a46db4b4197a093d, f09f152a68f54c518c0c8e931fed5941; no voters:
I20240502 13:59:44.981307 25408 raft_consensus.cc:2781] T 241895b448ae436b9b527a94c3331468 P 54fbce87fe064ab5a46db4b4197a093d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:44.981388 25408 raft_consensus.cc:686] T 241895b448ae436b9b527a94c3331468 P 54fbce87fe064ab5a46db4b4197a093d [term 1 LEADER]: Becoming Leader. State: Replica: 54fbce87fe064ab5a46db4b4197a093d, State: Running, Role: LEADER
I20240502 13:59:44.981549 25408 consensus_queue.cc:237] T 241895b448ae436b9b527a94c3331468 P 54fbce87fe064ab5a46db4b4197a093d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } }
I20240502 13:59:44.983031 24884 catalog_manager.cc:5484] T 241895b448ae436b9b527a94c3331468 P 54fbce87fe064ab5a46db4b4197a093d reported cstate change: term changed from 0 to 1, leader changed from <none> to 54fbce87fe064ab5a46db4b4197a093d (127.24.66.65). New cstate: current_term: 1 leader_uuid: "54fbce87fe064ab5a46db4b4197a093d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 } health_report { overall_health: UNKNOWN } } }
I20240502 13:59:45.001186 25404 consensus_queue.cc:1035] T cac3bbd142444b378b729de5cea6d1ae P 45c7ba84b0bb4626af59a8c46dcbb9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.005185 25404 consensus_queue.cc:1035] T cac3bbd142444b378b729de5cea6d1ae P 45c7ba84b0bb4626af59a8c46dcbb9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.035329 25408 consensus_queue.cc:1035] T 68513fdc8f5745088711a271afdbfe68 P 54fbce87fe064ab5a46db4b4197a093d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.038839 25408 consensus_queue.cc:1035] T 68513fdc8f5745088711a271afdbfe68 P 54fbce87fe064ab5a46db4b4197a093d [LEADER]: Connected to new peer: Peer: permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.076944 25408 consensus_queue.cc:1035] T 65bd1232d19e4a2c903fa19ede36a9ba P 54fbce87fe064ab5a46db4b4197a093d [LEADER]: Connected to new peer: Peer: permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.076951 25404 consensus_queue.cc:1035] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.082088 25412 consensus_queue.cc:1035] T 65bd1232d19e4a2c903fa19ede36a9ba P 54fbce87fe064ab5a46db4b4197a093d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.082126 25404 consensus_queue.cc:1035] T d7b93d774c9d45858e14ed8fd6cc5bf2 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.104458 25404 consensus_queue.cc:1035] T 758640af315340bd8d9e919888d30c86 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.107496 25402 consensus_queue.cc:1035] T 758640af315340bd8d9e919888d30c86 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.108736 25412 consensus_queue.cc:1035] T c8a7d300caa44b56918dc7045722e3c1 P 54fbce87fe064ab5a46db4b4197a093d [LEADER]: Connected to new peer: Peer: permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.113363 25412 consensus_queue.cc:1035] T c8a7d300caa44b56918dc7045722e3c1 P 54fbce87fe064ab5a46db4b4197a093d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.118857 25408 consensus_queue.cc:1035] T 5617b9f3b6bb436e88a2aeaf051c4700 P 54fbce87fe064ab5a46db4b4197a093d [LEADER]: Connected to new peer: Peer: permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.123612 25412 consensus_queue.cc:1035] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 54fbce87fe064ab5a46db4b4197a093d [LEADER]: Connected to new peer: Peer: permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.127805 25408 consensus_queue.cc:1035] T 5617b9f3b6bb436e88a2aeaf051c4700 P 54fbce87fe064ab5a46db4b4197a093d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.128672 25408 consensus_queue.cc:1035] T 2dbde6a2cb244667b7f24b6c4cb2c376 P 54fbce87fe064ab5a46db4b4197a093d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.155037 25408 consensus_queue.cc:1035] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.158990 25412 consensus_queue.cc:1035] T b99e4b9853404fe188ec39b5d2a6c154 P 54fbce87fe064ab5a46db4b4197a093d [LEADER]: Connected to new peer: Peer: permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.160248 25362 consensus_queue.cc:1035] T 34007af599ef4691977c361f01f22767 P f09f152a68f54c518c0c8e931fed5941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.181437 25362 consensus_queue.cc:1035] T 34007af599ef4691977c361f01f22767 P f09f152a68f54c518c0c8e931fed5941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.189095 25404 consensus_queue.cc:1035] T 1c75f6e566944a118ff217926b8b1f72 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.189244 25402 consensus_queue.cc:1035] T 1c75f6e566944a118ff217926b8b1f72 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.193183 25464 consensus_queue.cc:1035] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.196367 25362 consensus_queue.cc:1035] T 0bd322784404440983b3ca1026f13c10 P f09f152a68f54c518c0c8e931fed5941 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.198691 25391 consensus_queue.cc:1035] T 2b5e884f76de4e749872a7b9a2bd8493 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.206545 25391 consensus_queue.cc:1035] T 2b5e884f76de4e749872a7b9a2bd8493 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 13:59:45.218348 25451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:45.218607 25451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:45.221655 25451 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240502 13:59:45.315229 25391 consensus_queue.cc:1035] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.325747 25404 consensus_queue.cc:1035] T 8b30b6eada4345e8a8bc4e7bda88c8d5 P 45c7ba84b0bb4626af59a8c46dcbb9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.347438 25402 consensus_queue.cc:1035] T adc3311b364245c69107c0cef3c5ce8a P 45c7ba84b0bb4626af59a8c46dcbb9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.361995 25391 consensus_queue.cc:1035] T adc3311b364245c69107c0cef3c5ce8a P 45c7ba84b0bb4626af59a8c46dcbb9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54fbce87fe064ab5a46db4b4197a093d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 34575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.404793 25360 consensus_queue.cc:1035] T 241895b448ae436b9b527a94c3331468 P 54fbce87fe064ab5a46db4b4197a093d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f09f152a68f54c518c0c8e931fed5941" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 39311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:45.431279 25412 consensus_queue.cc:1035] T 241895b448ae436b9b527a94c3331468 P 54fbce87fe064ab5a46db4b4197a093d [LEADER]: Connected to new peer: Peer: permanent_uuid: "45c7ba84b0bb4626af59a8c46dcbb9c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 212 ms
time per row: 0.106 ms
I20240502 13:59:45.686632 25138 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 13:59:45.687269 25269 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:45.687297 25004 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 13:59:45.797065 25138 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 13:59:45.797785 25004 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 13:59:45.798255 25269 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:45.903990 25138 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 13:59:45.904258 25269 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:45.904762 25004 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 13:59:46.131419 25138 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 13:59:46.131867 25004 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 13:59:46.134074 25269 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:46.224663 25004 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 13:59:46.225286 25138 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 13:59:46.225292 25269 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:46.259999 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 24945
I20240502 13:59:46.266906 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 25077
I20240502 13:59:46.273502 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 25208
I20240502 13:59:46.279917 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 24853
2024-05-02T13:59:46Z chronyd exiting
[ OK ] TableRangeRebalancingTest.Basic (6373 ms)
[----------] 1 test from TableRangeRebalancingTest (6374 ms total)
[----------] 1 test from RebalanceStartSafetyTest
[ RUN ] RebalanceStartSafetyTest.TooManyIgnoredTservers/1
I20240502 13:59:46.310142 24841 test_util.cc:268] Using random seed: 67564536
I20240502 13:59:46.310909 24841 ts_itest-base.cc:112] Starting cluster with:
I20240502 13:59:46.310948 24841 ts_itest-base.cc:113] --------------
I20240502 13:59:46.310959 24841 ts_itest-base.cc:114] 5 tablet servers
I20240502 13:59:46.310971 24841 ts_itest-base.cc:115] 3 replicas per TS
I20240502 13:59:46.310981 24841 ts_itest-base.cc:116] --------------
2024-05-02T13:59:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T13:59:46Z Disabled control of system clock
I20240502 13:59:46.318863 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:33431
--webserver_interface=127.24.66.126
--webserver_port=0
--builtin_ntp_servers=127.24.66.84:45435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:33431
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false with env {}
W20240502 13:59:46.425223 25692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:46.425468 25692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:46.425539 25692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:46.428614 25692 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:46.428697 25692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 13:59:46.428730 25692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:46.428773 25692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 13:59:46.428808 25692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 13:59:46.432497 25692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:45435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:33431
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:33431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.25692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:46.433416 25692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:46.434568 25692 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:46.442337 25700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:46.442380 25697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:46.443213 25698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:46.443828 25692 server_base.cc:1034] running on GCE node
I20240502 13:59:46.444392 25692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:46.445371 25692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:46.446528 25692 hybrid_clock.cc:648] HybridClock initialized: now 1714658386446519 us; error 25 us; skew 500 ppm
I20240502 13:59:46.448714 25692 webserver.cc:460] Webserver started at http://127.24.66.126:37433/ using document root <none> and password file <none>
I20240502 13:59:46.449297 25692 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:46.449354 25692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:46.449551 25692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:46.451241 25692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "918cb930dfc04703874c0fe35f0af9cd"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-5npg"
I20240502 13:59:46.451745 25692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "918cb930dfc04703874c0fe35f0af9cd"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-5npg"
I20240502 13:59:46.455176 25692 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20240502 13:59:46.457433 25706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:46.458460 25692 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:46.458590 25692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
uuid: "918cb930dfc04703874c0fe35f0af9cd"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-5npg"
I20240502 13:59:46.458683 25692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:46.478089 25692 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:46.478749 25692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:46.478915 25692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:46.485330 25692 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:33431
I20240502 13:59:46.485342 25758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:33431 every 8 connection(s)
I20240502 13:59:46.486312 25692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 13:59:46.488806 25759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:46.493683 25759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 918cb930dfc04703874c0fe35f0af9cd: Bootstrap starting.
I20240502 13:59:46.494282 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 25692
I20240502 13:59:46.494388 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 13:59:46.496070 25759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 918cb930dfc04703874c0fe35f0af9cd: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:46.496752 25759 log.cc:824] T 00000000000000000000000000000000 P 918cb930dfc04703874c0fe35f0af9cd: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:46.498389 25759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 918cb930dfc04703874c0fe35f0af9cd: No bootstrap required, opened a new log
I20240502 13:59:46.501315 25759 raft_consensus.cc:348] T 00000000000000000000000000000000 P 918cb930dfc04703874c0fe35f0af9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918cb930dfc04703874c0fe35f0af9cd" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 33431 } }
I20240502 13:59:46.501495 25759 raft_consensus.cc:374] T 00000000000000000000000000000000 P 918cb930dfc04703874c0fe35f0af9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:46.501549 25759 raft_consensus.cc:729] T 00000000000000000000000000000000 P 918cb930dfc04703874c0fe35f0af9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 918cb930dfc04703874c0fe35f0af9cd, State: Initialized, Role: FOLLOWER
I20240502 13:59:46.501955 25759 consensus_queue.cc:260] T 00000000000000000000000000000000 P 918cb930dfc04703874c0fe35f0af9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918cb930dfc04703874c0fe35f0af9cd" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 33431 } }
I20240502 13:59:46.502077 25759 raft_consensus.cc:388] T 00000000000000000000000000000000 P 918cb930dfc04703874c0fe35f0af9cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 13:59:46.502178 25759 raft_consensus.cc:482] T 00000000000000000000000000000000 P 918cb930dfc04703874c0fe35f0af9cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 13:59:46.502293 25759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 918cb930dfc04703874c0fe35f0af9cd [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:46.503167 25759 raft_consensus.cc:504] T 00000000000000000000000000000000 P 918cb930dfc04703874c0fe35f0af9cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918cb930dfc04703874c0fe35f0af9cd" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 33431 } }
I20240502 13:59:46.503465 25759 leader_election.cc:302] T 00000000000000000000000000000000 P 918cb930dfc04703874c0fe35f0af9cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 918cb930dfc04703874c0fe35f0af9cd; no voters:
I20240502 13:59:46.503718 25759 leader_election.cc:288] T 00000000000000000000000000000000 P 918cb930dfc04703874c0fe35f0af9cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 13:59:46.503803 25765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 918cb930dfc04703874c0fe35f0af9cd [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:46.503999 25765 raft_consensus.cc:686] T 00000000000000000000000000000000 P 918cb930dfc04703874c0fe35f0af9cd [term 1 LEADER]: Becoming Leader. State: Replica: 918cb930dfc04703874c0fe35f0af9cd, State: Running, Role: LEADER
I20240502 13:59:46.504310 25765 consensus_queue.cc:237] T 00000000000000000000000000000000 P 918cb930dfc04703874c0fe35f0af9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918cb930dfc04703874c0fe35f0af9cd" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 33431 } }
I20240502 13:59:46.504750 25759 sys_catalog.cc:564] T 00000000000000000000000000000000 P 918cb930dfc04703874c0fe35f0af9cd [sys.catalog]: configured and running, proceeding with master startup.
I20240502 13:59:46.505805 25766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 918cb930dfc04703874c0fe35f0af9cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "918cb930dfc04703874c0fe35f0af9cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918cb930dfc04703874c0fe35f0af9cd" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 33431 } } }
I20240502 13:59:46.505836 25767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 918cb930dfc04703874c0fe35f0af9cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 918cb930dfc04703874c0fe35f0af9cd. Latest consensus state: current_term: 1 leader_uuid: "918cb930dfc04703874c0fe35f0af9cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "918cb930dfc04703874c0fe35f0af9cd" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 33431 } } }
I20240502 13:59:46.506026 25766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 918cb930dfc04703874c0fe35f0af9cd [sys.catalog]: This master's current role is: LEADER
I20240502 13:59:46.506036 25767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 918cb930dfc04703874c0fe35f0af9cd [sys.catalog]: This master's current role is: LEADER
I20240502 13:59:46.506783 25774 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 13:59:46.509369 25774 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 13:59:46.514504 25774 catalog_manager.cc:1347] Generated new cluster ID: a812f9c439794c68b08d4f8155355a90
I20240502 13:59:46.514566 25774 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 13:59:46.522295 25774 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 13:59:46.523115 25774 catalog_manager.cc:1504] Loading token signing keys...
I20240502 13:59:46.530375 25774 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 918cb930dfc04703874c0fe35f0af9cd: Generated new TSK 0
I20240502 13:59:46.531060 25774 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 13:59:46.535905 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:33431
--builtin_ntp_servers=127.24.66.84:45435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 13:59:46.642414 25784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:46.642632 25784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:46.642699 25784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:46.645733 25784 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:46.645808 25784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:46.645896 25784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20240502 13:59:46.649413 25784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:45435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:33431
--never_fsync=true
--heap_profile_path=/tmp/kudu.25784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:46.650231 25784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:46.651316 25784 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:46.659298 25790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:46.659871 25792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:46.660660 25789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:46.661317 25784 server_base.cc:1034] running on GCE node
I20240502 13:59:46.661710 25784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:46.662231 25784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:46.663386 25784 hybrid_clock.cc:648] HybridClock initialized: now 1714658386663364 us; error 39 us; skew 500 ppm
I20240502 13:59:46.665201 25784 webserver.cc:460] Webserver started at http://127.24.66.65:37689/ using document root <none> and password file <none>
I20240502 13:59:46.665724 25784 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:46.665781 25784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:46.665949 25784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:46.667531 25784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "db353c5de884450fb303d38cfa148b93"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-5npg"
I20240502 13:59:46.667984 25784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "db353c5de884450fb303d38cfa148b93"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-5npg"
I20240502 13:59:46.671237 25784 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 13:59:46.673305 25798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:46.674177 25784 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:46.674273 25784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "db353c5de884450fb303d38cfa148b93"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-5npg"
I20240502 13:59:46.674346 25784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:46.696907 25784 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:46.697489 25784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:46.697644 25784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:46.698134 25784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:46.698959 25784 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:46.699005 25784 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:46.699060 25784 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:46.699088 25784 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:46.707017 25784 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:46757
I20240502 13:59:46.707067 25912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:46757 every 8 connection(s)
I20240502 13:59:46.707947 25784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 13:59:46.711432 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 25784
I20240502 13:59:46.711521 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 13:59:46.713685 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:33431
--builtin_ntp_servers=127.24.66.84:45435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 13:59:46.717805 25913 heartbeater.cc:347] Connected to a master server at 127.24.66.126:33431
I20240502 13:59:46.718003 25913 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:46.718469 25913 heartbeater.cc:511] Master 127.24.66.126:33431 requested a full tablet report, sending...
I20240502 13:59:46.719797 25723 ts_manager.cc:194] Registered new tserver with Master: db353c5de884450fb303d38cfa148b93 (127.24.66.65:46757)
I20240502 13:59:46.720911 25723 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.65:34331
W20240502 13:59:46.817597 25916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:46.817801 25916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:46.817870 25916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:46.820748 25916 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:46.820814 25916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:46.820895 25916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20240502 13:59:46.824796 25916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:45435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:33431
--never_fsync=true
--heap_profile_path=/tmp/kudu.25916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:46.825610 25916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:46.826606 25916 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:46.834264 25924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:46.834897 25922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:46.835476 25921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:46.836357 25916 server_base.cc:1034] running on GCE node
I20240502 13:59:46.836731 25916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:46.837239 25916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:46.838387 25916 hybrid_clock.cc:648] HybridClock initialized: now 1714658386838369 us; error 31 us; skew 500 ppm
I20240502 13:59:46.840227 25916 webserver.cc:460] Webserver started at http://127.24.66.66:33675/ using document root <none> and password file <none>
I20240502 13:59:46.840756 25916 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:46.840814 25916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:46.841002 25916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:46.842568 25916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "8ed1f4b6a64344358a13ffae5def90c1"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-5npg"
I20240502 13:59:46.843014 25916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "8ed1f4b6a64344358a13ffae5def90c1"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-5npg"
I20240502 13:59:46.846153 25916 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 13:59:46.848443 25930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:46.849372 25916 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 13:59:46.849470 25916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "8ed1f4b6a64344358a13ffae5def90c1"
format_stamp: "Formatted at 2024-05-02 13:59:46 on dist-test-slave-5npg"
I20240502 13:59:46.849553 25916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:46.870690 25916 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:46.871273 25916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:46.871433 25916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:46.871956 25916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:46.872903 25916 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:46.872951 25916 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:46.873008 25916 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:46.873039 25916 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:46.881178 25916 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:40331
I20240502 13:59:46.881235 26044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:40331 every 8 connection(s)
I20240502 13:59:46.882056 25916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 13:59:46.889614 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 25916
I20240502 13:59:46.889708 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 13:59:46.890959 26045 heartbeater.cc:347] Connected to a master server at 127.24.66.126:33431
I20240502 13:59:46.891176 26045 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:46.891695 26045 heartbeater.cc:511] Master 127.24.66.126:33431 requested a full tablet report, sending...
I20240502 13:59:46.891878 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:33431
--builtin_ntp_servers=127.24.66.84:45435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 13:59:46.892596 25722 ts_manager.cc:194] Registered new tserver with Master: 8ed1f4b6a64344358a13ffae5def90c1 (127.24.66.66:40331)
I20240502 13:59:46.893121 25722 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.66:33489
W20240502 13:59:46.995380 26048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:46.995616 26048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:46.995663 26048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:46.998682 26048 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:46.998741 26048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:46.998811 26048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20240502 13:59:47.002453 26048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:45435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:33431
--never_fsync=true
--heap_profile_path=/tmp/kudu.26048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:47.003275 26048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:47.004371 26048 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:47.012305 26054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:47.012972 26056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:47.013643 26053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:47.014369 26048 server_base.cc:1034] running on GCE node
I20240502 13:59:47.014725 26048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:47.015239 26048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:47.016386 26048 hybrid_clock.cc:648] HybridClock initialized: now 1714658387016373 us; error 33 us; skew 500 ppm
I20240502 13:59:47.018182 26048 webserver.cc:460] Webserver started at http://127.24.66.67:36877/ using document root <none> and password file <none>
I20240502 13:59:47.018719 26048 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:47.018779 26048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:47.018944 26048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:47.020529 26048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "aa16690bcac448ea80b337fafe1c01e1"
format_stamp: "Formatted at 2024-05-02 13:59:47 on dist-test-slave-5npg"
I20240502 13:59:47.020967 26048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "aa16690bcac448ea80b337fafe1c01e1"
format_stamp: "Formatted at 2024-05-02 13:59:47 on dist-test-slave-5npg"
I20240502 13:59:47.024163 26048 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 13:59:47.026139 26062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:47.027029 26048 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:47.027127 26048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "aa16690bcac448ea80b337fafe1c01e1"
format_stamp: "Formatted at 2024-05-02 13:59:47 on dist-test-slave-5npg"
I20240502 13:59:47.027202 26048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:47.037117 26048 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:47.037601 26048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:47.037739 26048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:47.038168 26048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:47.038931 26048 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:47.038985 26048 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:47.039036 26048 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:47.039062 26048 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:47.046540 26048 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:32815
I20240502 13:59:47.046563 26176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:32815 every 8 connection(s)
I20240502 13:59:47.047345 26048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 13:59:47.047727 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 26048
I20240502 13:59:47.047825 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 13:59:47.049976 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.68:0
--local_ip_for_outbound_sockets=127.24.66.68
--webserver_interface=127.24.66.68
--webserver_port=0
--tserver_master_addrs=127.24.66.126:33431
--builtin_ntp_servers=127.24.66.84:45435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 13:59:47.055954 26177 heartbeater.cc:347] Connected to a master server at 127.24.66.126:33431
I20240502 13:59:47.056130 26177 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:47.056636 26177 heartbeater.cc:511] Master 127.24.66.126:33431 requested a full tablet report, sending...
I20240502 13:59:47.057571 25722 ts_manager.cc:194] Registered new tserver with Master: aa16690bcac448ea80b337fafe1c01e1 (127.24.66.67:32815)
I20240502 13:59:47.058079 25722 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.67:44093
W20240502 13:59:47.150961 26180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:47.151166 26180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:47.151239 26180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:47.154386 26180 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:47.154578 26180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:47.154771 26180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.68
I20240502 13:59:47.158430 26180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:45435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.24.66.68
--webserver_port=0
--tserver_master_addrs=127.24.66.126:33431
--never_fsync=true
--heap_profile_path=/tmp/kudu.26180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.68
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:47.159242 26180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:47.160295 26180 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:47.167966 26188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:47.168706 26185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:47.169240 26186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:47.170112 26180 server_base.cc:1034] running on GCE node
I20240502 13:59:47.170459 26180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:47.170979 26180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:47.172134 26180 hybrid_clock.cc:648] HybridClock initialized: now 1714658387172120 us; error 37 us; skew 500 ppm
I20240502 13:59:47.173980 26180 webserver.cc:460] Webserver started at http://127.24.66.68:39151/ using document root <none> and password file <none>
I20240502 13:59:47.174489 26180 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:47.174556 26180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:47.174719 26180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:47.176260 26180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "7c90e23549bd4f36910a4f6703c93479"
format_stamp: "Formatted at 2024-05-02 13:59:47 on dist-test-slave-5npg"
I20240502 13:59:47.176744 26180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "7c90e23549bd4f36910a4f6703c93479"
format_stamp: "Formatted at 2024-05-02 13:59:47 on dist-test-slave-5npg"
I20240502 13:59:47.179836 26180 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 13:59:47.181901 26194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:47.182804 26180 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20240502 13:59:47.182909 26180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "7c90e23549bd4f36910a4f6703c93479"
format_stamp: "Formatted at 2024-05-02 13:59:47 on dist-test-slave-5npg"
I20240502 13:59:47.182987 26180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:47.200264 26180 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:47.200850 26180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:47.200996 26180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:47.201467 26180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:47.202275 26180 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:47.202324 26180 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:47.202383 26180 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:47.202410 26180 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:47.210745 26180 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.68:37443
I20240502 13:59:47.210788 26308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.68:37443 every 8 connection(s)
I20240502 13:59:47.211647 26180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 13:59:47.215526 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 26180
I20240502 13:59:47.215626 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 13:59:47.217756 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.69:0
--local_ip_for_outbound_sockets=127.24.66.69
--webserver_interface=127.24.66.69
--webserver_port=0
--tserver_master_addrs=127.24.66.126:33431
--builtin_ntp_servers=127.24.66.84:45435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 13:59:47.220443 26309 heartbeater.cc:347] Connected to a master server at 127.24.66.126:33431
I20240502 13:59:47.220647 26309 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:47.221175 26309 heartbeater.cc:511] Master 127.24.66.126:33431 requested a full tablet report, sending...
I20240502 13:59:47.222146 25722 ts_manager.cc:194] Registered new tserver with Master: 7c90e23549bd4f36910a4f6703c93479 (127.24.66.68:37443)
I20240502 13:59:47.222966 25722 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.68:42593
W20240502 13:59:47.317875 26312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:47.318068 26312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:47.318110 26312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:47.320991 26312 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:47.321045 26312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:47.321106 26312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.69
I20240502 13:59:47.324584 26312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:45435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.69:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.24.66.69
--webserver_port=0
--tserver_master_addrs=127.24.66.126:33431
--never_fsync=true
--heap_profile_path=/tmp/kudu.26312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.69
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:47.325322 26312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:47.326332 26312 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:47.333792 26317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:47.334446 26312 server_base.cc:1034] running on GCE node
W20240502 13:59:47.335112 26318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:47.335698 26320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:47.336107 26312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:47.336676 26312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:47.337831 26312 hybrid_clock.cc:648] HybridClock initialized: now 1714658387337804 us; error 39 us; skew 500 ppm
I20240502 13:59:47.339685 26312 webserver.cc:460] Webserver started at http://127.24.66.69:34163/ using document root <none> and password file <none>
I20240502 13:59:47.340188 26312 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:47.340238 26312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:47.340433 26312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:47.342044 26312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "5bee4725e48b4beca05bb6ef80e3b18b"
format_stamp: "Formatted at 2024-05-02 13:59:47 on dist-test-slave-5npg"
I20240502 13:59:47.342459 26312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "5bee4725e48b4beca05bb6ef80e3b18b"
format_stamp: "Formatted at 2024-05-02 13:59:47 on dist-test-slave-5npg"
I20240502 13:59:47.345705 26312 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 13:59:47.348031 26326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:47.349154 26312 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 13:59:47.349267 26312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "5bee4725e48b4beca05bb6ef80e3b18b"
format_stamp: "Formatted at 2024-05-02 13:59:47 on dist-test-slave-5npg"
I20240502 13:59:47.349370 26312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:47.370280 26312 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:47.370945 26312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:47.371119 26312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:47.371706 26312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:47.372694 26312 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:47.372745 26312 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:47.372790 26312 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:47.372822 26312 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:47.381783 26312 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.69:37743
I20240502 13:59:47.381844 26439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.69:37743 every 8 connection(s)
I20240502 13:59:47.382938 26312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 13:59:47.383424 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 26312
I20240502 13:59:47.383538 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceStartSafetyTest.TooManyIgnoredTservers_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 13:59:47.391927 26440 heartbeater.cc:347] Connected to a master server at 127.24.66.126:33431
I20240502 13:59:47.392143 26440 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:47.392694 26440 heartbeater.cc:511] Master 127.24.66.126:33431 requested a full tablet report, sending...
I20240502 13:59:47.393604 25722 ts_manager.cc:194] Registered new tserver with Master: 5bee4725e48b4beca05bb6ef80e3b18b (127.24.66.69:37743)
I20240502 13:59:47.394134 25722 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.69:55923
I20240502 13:59:47.397926 24841 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240502 13:59:47.406214 25722 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50938:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240502 13:59:47.423615 25978 tablet_service.cc:1467] Processing CreateTablet for tablet e436ae831aa7446393d4c59a23b1bd23 (DEFAULT_TABLE table=TestTable [id=af738fed5a69490783acb27e3f6ef4e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:47.423707 25846 tablet_service.cc:1467] Processing CreateTablet for tablet e436ae831aa7446393d4c59a23b1bd23 (DEFAULT_TABLE table=TestTable [id=af738fed5a69490783acb27e3f6ef4e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:47.423707 26242 tablet_service.cc:1467] Processing CreateTablet for tablet e436ae831aa7446393d4c59a23b1bd23 (DEFAULT_TABLE table=TestTable [id=af738fed5a69490783acb27e3f6ef4e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:47.424569 25978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e436ae831aa7446393d4c59a23b1bd23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:47.424592 26242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e436ae831aa7446393d4c59a23b1bd23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:47.424592 25846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e436ae831aa7446393d4c59a23b1bd23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:47.429363 26457 tablet_bootstrap.cc:492] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1: Bootstrap starting.
I20240502 13:59:47.430527 26459 tablet_bootstrap.cc:492] T e436ae831aa7446393d4c59a23b1bd23 P 7c90e23549bd4f36910a4f6703c93479: Bootstrap starting.
I20240502 13:59:47.430802 26458 tablet_bootstrap.cc:492] T e436ae831aa7446393d4c59a23b1bd23 P db353c5de884450fb303d38cfa148b93: Bootstrap starting.
I20240502 13:59:47.431535 26457 tablet_bootstrap.cc:654] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:47.432379 26457 log.cc:824] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:47.432572 26458 tablet_bootstrap.cc:654] T e436ae831aa7446393d4c59a23b1bd23 P db353c5de884450fb303d38cfa148b93: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:47.432572 26459 tablet_bootstrap.cc:654] T e436ae831aa7446393d4c59a23b1bd23 P 7c90e23549bd4f36910a4f6703c93479: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:47.433338 26458 log.cc:824] T e436ae831aa7446393d4c59a23b1bd23 P db353c5de884450fb303d38cfa148b93: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:47.433338 26459 log.cc:824] T e436ae831aa7446393d4c59a23b1bd23 P 7c90e23549bd4f36910a4f6703c93479: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:47.433928 26457 tablet_bootstrap.cc:492] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1: No bootstrap required, opened a new log
I20240502 13:59:47.434108 26457 ts_tablet_manager.cc:1367] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240502 13:59:47.434939 26459 tablet_bootstrap.cc:492] T e436ae831aa7446393d4c59a23b1bd23 P 7c90e23549bd4f36910a4f6703c93479: No bootstrap required, opened a new log
I20240502 13:59:47.435134 26459 ts_tablet_manager.cc:1367] T e436ae831aa7446393d4c59a23b1bd23 P 7c90e23549bd4f36910a4f6703c93479: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 13:59:47.435151 26458 tablet_bootstrap.cc:492] T e436ae831aa7446393d4c59a23b1bd23 P db353c5de884450fb303d38cfa148b93: No bootstrap required, opened a new log
I20240502 13:59:47.435282 26458 ts_tablet_manager.cc:1367] T e436ae831aa7446393d4c59a23b1bd23 P db353c5de884450fb303d38cfa148b93: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240502 13:59:47.436975 26457 raft_consensus.cc:348] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c90e23549bd4f36910a4f6703c93479" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37443 } } peers { permanent_uuid: "db353c5de884450fb303d38cfa148b93" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46757 } } peers { permanent_uuid: "8ed1f4b6a64344358a13ffae5def90c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40331 } }
I20240502 13:59:47.437152 26457 raft_consensus.cc:374] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:47.437227 26457 raft_consensus.cc:729] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ed1f4b6a64344358a13ffae5def90c1, State: Initialized, Role: FOLLOWER
I20240502 13:59:47.437712 26457 consensus_queue.cc:260] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c90e23549bd4f36910a4f6703c93479" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37443 } } peers { permanent_uuid: "db353c5de884450fb303d38cfa148b93" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46757 } } peers { permanent_uuid: "8ed1f4b6a64344358a13ffae5def90c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40331 } }
I20240502 13:59:47.438426 26045 heartbeater.cc:503] Master 127.24.66.126:33431 was elected leader, sending a full tablet report...
I20240502 13:59:47.438659 26457 ts_tablet_manager.cc:1398] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20240502 13:59:47.438838 26459 raft_consensus.cc:348] T e436ae831aa7446393d4c59a23b1bd23 P 7c90e23549bd4f36910a4f6703c93479 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c90e23549bd4f36910a4f6703c93479" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37443 } } peers { permanent_uuid: "db353c5de884450fb303d38cfa148b93" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46757 } } peers { permanent_uuid: "8ed1f4b6a64344358a13ffae5def90c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40331 } }
I20240502 13:59:47.438838 26458 raft_consensus.cc:348] T e436ae831aa7446393d4c59a23b1bd23 P db353c5de884450fb303d38cfa148b93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c90e23549bd4f36910a4f6703c93479" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37443 } } peers { permanent_uuid: "db353c5de884450fb303d38cfa148b93" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46757 } } peers { permanent_uuid: "8ed1f4b6a64344358a13ffae5def90c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40331 } }
I20240502 13:59:47.439031 26459 raft_consensus.cc:374] T e436ae831aa7446393d4c59a23b1bd23 P 7c90e23549bd4f36910a4f6703c93479 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:47.439031 26458 raft_consensus.cc:374] T e436ae831aa7446393d4c59a23b1bd23 P db353c5de884450fb303d38cfa148b93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:47.439095 26458 raft_consensus.cc:729] T e436ae831aa7446393d4c59a23b1bd23 P db353c5de884450fb303d38cfa148b93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db353c5de884450fb303d38cfa148b93, State: Initialized, Role: FOLLOWER
I20240502 13:59:47.439137 26459 raft_consensus.cc:729] T e436ae831aa7446393d4c59a23b1bd23 P 7c90e23549bd4f36910a4f6703c93479 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c90e23549bd4f36910a4f6703c93479, State: Initialized, Role: FOLLOWER
I20240502 13:59:47.439522 26458 consensus_queue.cc:260] T e436ae831aa7446393d4c59a23b1bd23 P db353c5de884450fb303d38cfa148b93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c90e23549bd4f36910a4f6703c93479" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37443 } } peers { permanent_uuid: "db353c5de884450fb303d38cfa148b93" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46757 } } peers { permanent_uuid: "8ed1f4b6a64344358a13ffae5def90c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40331 } }
I20240502 13:59:47.439522 26459 consensus_queue.cc:260] T e436ae831aa7446393d4c59a23b1bd23 P 7c90e23549bd4f36910a4f6703c93479 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c90e23549bd4f36910a4f6703c93479" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37443 } } peers { permanent_uuid: "db353c5de884450fb303d38cfa148b93" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46757 } } peers { permanent_uuid: "8ed1f4b6a64344358a13ffae5def90c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40331 } }
I20240502 13:59:47.440304 26459 ts_tablet_manager.cc:1398] T e436ae831aa7446393d4c59a23b1bd23 P 7c90e23549bd4f36910a4f6703c93479: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240502 13:59:47.440304 26458 ts_tablet_manager.cc:1398] T e436ae831aa7446393d4c59a23b1bd23 P db353c5de884450fb303d38cfa148b93: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20240502 13:59:47.440363 26309 heartbeater.cc:503] Master 127.24.66.126:33431 was elected leader, sending a full tablet report...
I20240502 13:59:47.441291 25913 heartbeater.cc:503] Master 127.24.66.126:33431 was elected leader, sending a full tablet report...
W20240502 13:59:47.459124 25914 tablet.cc:2369] T e436ae831aa7446393d4c59a23b1bd23 P db353c5de884450fb303d38cfa148b93: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 13:59:47.462400 26310 tablet.cc:2369] T e436ae831aa7446393d4c59a23b1bd23 P 7c90e23549bd4f36910a4f6703c93479: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 13:59:47.563370 26463 raft_consensus.cc:482] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:47.563581 26463 raft_consensus.cc:504] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c90e23549bd4f36910a4f6703c93479" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37443 } } peers { permanent_uuid: "db353c5de884450fb303d38cfa148b93" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46757 } } peers { permanent_uuid: "8ed1f4b6a64344358a13ffae5def90c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40331 } }
I20240502 13:59:47.564805 26463 leader_election.cc:288] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c90e23549bd4f36910a4f6703c93479 (127.24.66.68:37443), db353c5de884450fb303d38cfa148b93 (127.24.66.65:46757)
I20240502 13:59:47.568585 26262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e436ae831aa7446393d4c59a23b1bd23" candidate_uuid: "8ed1f4b6a64344358a13ffae5def90c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c90e23549bd4f36910a4f6703c93479" is_pre_election: true
I20240502 13:59:47.568926 26262 raft_consensus.cc:2454] T e436ae831aa7446393d4c59a23b1bd23 P 7c90e23549bd4f36910a4f6703c93479 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ed1f4b6a64344358a13ffae5def90c1 in term 0.
I20240502 13:59:47.569061 25866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e436ae831aa7446393d4c59a23b1bd23" candidate_uuid: "8ed1f4b6a64344358a13ffae5def90c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db353c5de884450fb303d38cfa148b93" is_pre_election: true
I20240502 13:59:47.569326 25866 raft_consensus.cc:2454] T e436ae831aa7446393d4c59a23b1bd23 P db353c5de884450fb303d38cfa148b93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ed1f4b6a64344358a13ffae5def90c1 in term 0.
I20240502 13:59:47.569366 25932 leader_election.cc:302] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c90e23549bd4f36910a4f6703c93479, 8ed1f4b6a64344358a13ffae5def90c1; no voters:
I20240502 13:59:47.569617 26463 raft_consensus.cc:2781] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:47.569715 26463 raft_consensus.cc:482] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:47.569772 26463 raft_consensus.cc:3037] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:47.570669 26463 raft_consensus.cc:504] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c90e23549bd4f36910a4f6703c93479" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37443 } } peers { permanent_uuid: "db353c5de884450fb303d38cfa148b93" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46757 } } peers { permanent_uuid: "8ed1f4b6a64344358a13ffae5def90c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40331 } }
I20240502 13:59:47.570961 26463 leader_election.cc:288] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1 [CANDIDATE]: Term 1 election: Requested vote from peers 7c90e23549bd4f36910a4f6703c93479 (127.24.66.68:37443), db353c5de884450fb303d38cfa148b93 (127.24.66.65:46757)
I20240502 13:59:47.571363 26262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e436ae831aa7446393d4c59a23b1bd23" candidate_uuid: "8ed1f4b6a64344358a13ffae5def90c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c90e23549bd4f36910a4f6703c93479"
I20240502 13:59:47.571466 26262 raft_consensus.cc:3037] T e436ae831aa7446393d4c59a23b1bd23 P 7c90e23549bd4f36910a4f6703c93479 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:47.571451 25866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e436ae831aa7446393d4c59a23b1bd23" candidate_uuid: "8ed1f4b6a64344358a13ffae5def90c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db353c5de884450fb303d38cfa148b93"
I20240502 13:59:47.571580 25866 raft_consensus.cc:3037] T e436ae831aa7446393d4c59a23b1bd23 P db353c5de884450fb303d38cfa148b93 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:47.572299 26262 raft_consensus.cc:2454] T e436ae831aa7446393d4c59a23b1bd23 P 7c90e23549bd4f36910a4f6703c93479 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ed1f4b6a64344358a13ffae5def90c1 in term 1.
I20240502 13:59:47.572441 25866 raft_consensus.cc:2454] T e436ae831aa7446393d4c59a23b1bd23 P db353c5de884450fb303d38cfa148b93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ed1f4b6a64344358a13ffae5def90c1 in term 1.
I20240502 13:59:47.572695 25932 leader_election.cc:302] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c90e23549bd4f36910a4f6703c93479, 8ed1f4b6a64344358a13ffae5def90c1; no voters:
I20240502 13:59:47.572855 26463 raft_consensus.cc:2781] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:47.573046 26463 raft_consensus.cc:686] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1 [term 1 LEADER]: Becoming Leader. State: Replica: 8ed1f4b6a64344358a13ffae5def90c1, State: Running, Role: LEADER
I20240502 13:59:47.573323 26463 consensus_queue.cc:237] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c90e23549bd4f36910a4f6703c93479" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37443 } } peers { permanent_uuid: "db353c5de884450fb303d38cfa148b93" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46757 } } peers { permanent_uuid: "8ed1f4b6a64344358a13ffae5def90c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40331 } }
I20240502 13:59:47.576217 25722 catalog_manager.cc:5484] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ed1f4b6a64344358a13ffae5def90c1 (127.24.66.66). New cstate: current_term: 1 leader_uuid: "8ed1f4b6a64344358a13ffae5def90c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c90e23549bd4f36910a4f6703c93479" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db353c5de884450fb303d38cfa148b93" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ed1f4b6a64344358a13ffae5def90c1" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40331 } health_report { overall_health: HEALTHY } } }
I20240502 13:59:47.605105 24841 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240502 13:59:47.606724 24841 ts_itest-base.cc:233] Waiting for 1 tablets on tserver db353c5de884450fb303d38cfa148b93 to finish bootstrapping
I20240502 13:59:47.611923 24841 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 8ed1f4b6a64344358a13ffae5def90c1 to finish bootstrapping
I20240502 13:59:47.616181 24841 ts_itest-base.cc:233] Waiting for 0 tablets on tserver aa16690bcac448ea80b337fafe1c01e1 to finish bootstrapping
I20240502 13:59:47.621179 24841 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 7c90e23549bd4f36910a4f6703c93479 to finish bootstrapping
I20240502 13:59:47.625514 24841 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 5bee4725e48b4beca05bb6ef80e3b18b to finish bootstrapping
W20240502 13:59:47.633244 26046 tablet.cc:2369] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 13:59:47.803095 26110 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:47.803877 26374 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:47.804262 25978 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:47.806972 25846 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:47.816942 26242 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:47.891700 26374 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:47.894148 26242 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:47.895934 26110 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:47.896466 25978 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:47.908092 25846 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:47.992617 26242 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:47.994251 25978 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:47.994472 26374 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:47.995028 26110 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:47.995720 25846 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:48.058205 26463 consensus_queue.cc:1035] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db353c5de884450fb303d38cfa148b93" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 46757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:48.059692 26177 heartbeater.cc:503] Master 127.24.66.126:33431 was elected leader, sending a full tablet report...
I20240502 13:59:48.063989 26575 consensus_queue.cc:1035] T e436ae831aa7446393d4c59a23b1bd23 P 8ed1f4b6a64344358a13ffae5def90c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c90e23549bd4f36910a4f6703c93479" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:48.204772 26110 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:48.205422 26374 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:48.206148 25978 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:48.211374 26242 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:48.212270 25846 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:48.295033 26242 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:48.295483 26110 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:48.295914 26374 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:48.296953 25978 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:48.298885 25846 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:48.396234 26440 heartbeater.cc:503] Master 127.24.66.126:33431 was elected leader, sending a full tablet report...
I20240502 13:59:48.397029 26242 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:48.397614 25978 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:48.398397 26110 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:48.401262 25846 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:48.409407 26374 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:48.436513 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 25784
I20240502 13:59:48.442277 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 25916
I20240502 13:59:48.447758 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 26048
I20240502 13:59:48.453089 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 26180
I20240502 13:59:48.458626 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 26312
I20240502 13:59:48.464071 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 25692
2024-05-02T13:59:48Z chronyd exiting
[ OK ] RebalanceStartSafetyTest.TooManyIgnoredTservers/1 (2176 ms)
[----------] 1 test from RebalanceStartSafetyTest (2176 ms total)
[----------] 1 test from RebalanceParamTest
[ RUN ] RebalanceParamTest.Rebalance/7
I20240502 13:59:48.486739 24841 test_util.cc:268] Using random seed: 69741134
I20240502 13:59:48.487488 24841 ts_itest-base.cc:112] Starting cluster with:
I20240502 13:59:48.487535 24841 ts_itest-base.cc:113] --------------
I20240502 13:59:48.487547 24841 ts_itest-base.cc:114] 7 tablet servers
I20240502 13:59:48.487557 24841 ts_itest-base.cc:115] 5 replicas per TS
I20240502 13:59:48.487568 24841 ts_itest-base.cc:116] --------------
2024-05-02T13:59:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T13:59:48Z Disabled control of system clock
I20240502 13:59:48.495172 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:45215
--webserver_interface=127.24.66.126
--webserver_port=0
--builtin_ntp_servers=127.24.66.84:37497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:45215
--allow_unsafe_replication_factor
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--tserver_unresponsive_timeout_ms=3000 with env {}
W20240502 13:59:48.598081 26685 flags.cc:425] Enabled unsafe flag: --allow_unsafe_replication_factor=true
W20240502 13:59:48.598354 26685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:48.598431 26685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:48.598505 26685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:48.602005 26685 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:48.602089 26685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 13:59:48.602134 26685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:48.602172 26685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 13:59:48.602207 26685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 13:59:48.605875 26685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:37497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
--allow_unsafe_replication_factor=true
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:45215
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:45215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.26685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:48.606792 26685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:48.607880 26685 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:48.615150 26690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:48.615118 26691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:48.615880 26693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:48.616528 26685 server_base.cc:1034] running on GCE node
I20240502 13:59:48.616981 26685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:48.618011 26685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:48.619179 26685 hybrid_clock.cc:648] HybridClock initialized: now 1714658388619077 us; error 111 us; skew 500 ppm
I20240502 13:59:48.621865 26685 webserver.cc:460] Webserver started at http://127.24.66.126:46683/ using document root <none> and password file <none>
I20240502 13:59:48.622715 26685 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:48.622802 26685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:48.623028 26685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:48.625290 26685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "60bad1ef960c4e09b6fb0fe11be17da2"
format_stamp: "Formatted at 2024-05-02 13:59:48 on dist-test-slave-5npg"
I20240502 13:59:48.625993 26685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "60bad1ef960c4e09b6fb0fe11be17da2"
format_stamp: "Formatted at 2024-05-02 13:59:48 on dist-test-slave-5npg"
I20240502 13:59:48.630597 26685 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20240502 13:59:48.633553 26699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:48.635025 26685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20240502 13:59:48.635149 26685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
uuid: "60bad1ef960c4e09b6fb0fe11be17da2"
format_stamp: "Formatted at 2024-05-02 13:59:48 on dist-test-slave-5npg"
I20240502 13:59:48.635259 26685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:48.665691 26685 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:48.666301 26685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:48.666432 26685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:48.672952 26752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:45215 every 8 connection(s)
I20240502 13:59:48.672952 26685 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:45215
I20240502 13:59:48.674041 26685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 13:59:48.676699 26753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:48.680248 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 26685
I20240502 13:59:48.680382 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 13:59:48.682323 26753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60bad1ef960c4e09b6fb0fe11be17da2: Bootstrap starting.
I20240502 13:59:48.684674 26753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60bad1ef960c4e09b6fb0fe11be17da2: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:48.685482 26753 log.cc:824] T 00000000000000000000000000000000 P 60bad1ef960c4e09b6fb0fe11be17da2: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:48.687357 26753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60bad1ef960c4e09b6fb0fe11be17da2: No bootstrap required, opened a new log
I20240502 13:59:48.689888 26753 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60bad1ef960c4e09b6fb0fe11be17da2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60bad1ef960c4e09b6fb0fe11be17da2" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45215 } }
I20240502 13:59:48.690066 26753 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60bad1ef960c4e09b6fb0fe11be17da2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:48.690129 26753 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60bad1ef960c4e09b6fb0fe11be17da2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60bad1ef960c4e09b6fb0fe11be17da2, State: Initialized, Role: FOLLOWER
I20240502 13:59:48.690491 26753 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60bad1ef960c4e09b6fb0fe11be17da2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60bad1ef960c4e09b6fb0fe11be17da2" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45215 } }
I20240502 13:59:48.690614 26753 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60bad1ef960c4e09b6fb0fe11be17da2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 13:59:48.690680 26753 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60bad1ef960c4e09b6fb0fe11be17da2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 13:59:48.690785 26753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60bad1ef960c4e09b6fb0fe11be17da2 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:48.701526 26753 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60bad1ef960c4e09b6fb0fe11be17da2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60bad1ef960c4e09b6fb0fe11be17da2" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45215 } }
I20240502 13:59:48.701934 26753 leader_election.cc:302] T 00000000000000000000000000000000 P 60bad1ef960c4e09b6fb0fe11be17da2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60bad1ef960c4e09b6fb0fe11be17da2; no voters:
I20240502 13:59:48.702248 26753 leader_election.cc:288] T 00000000000000000000000000000000 P 60bad1ef960c4e09b6fb0fe11be17da2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 13:59:48.702828 26758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60bad1ef960c4e09b6fb0fe11be17da2 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:48.703042 26758 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60bad1ef960c4e09b6fb0fe11be17da2 [term 1 LEADER]: Becoming Leader. State: Replica: 60bad1ef960c4e09b6fb0fe11be17da2, State: Running, Role: LEADER
I20240502 13:59:48.703378 26758 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60bad1ef960c4e09b6fb0fe11be17da2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60bad1ef960c4e09b6fb0fe11be17da2" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45215 } }
I20240502 13:59:48.703576 26753 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60bad1ef960c4e09b6fb0fe11be17da2 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 13:59:48.705226 26759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60bad1ef960c4e09b6fb0fe11be17da2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60bad1ef960c4e09b6fb0fe11be17da2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60bad1ef960c4e09b6fb0fe11be17da2" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45215 } } }
I20240502 13:59:48.705355 26759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60bad1ef960c4e09b6fb0fe11be17da2 [sys.catalog]: This master's current role is: LEADER
I20240502 13:59:48.705296 26760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60bad1ef960c4e09b6fb0fe11be17da2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60bad1ef960c4e09b6fb0fe11be17da2. Latest consensus state: current_term: 1 leader_uuid: "60bad1ef960c4e09b6fb0fe11be17da2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60bad1ef960c4e09b6fb0fe11be17da2" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 45215 } } }
I20240502 13:59:48.705422 26760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60bad1ef960c4e09b6fb0fe11be17da2 [sys.catalog]: This master's current role is: LEADER
I20240502 13:59:48.705794 26771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 13:59:48.708918 26771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 13:59:48.713945 26771 catalog_manager.cc:1347] Generated new cluster ID: 40613fb6dbcf46cda4d3f68ef2a442cf
I20240502 13:59:48.714017 26771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 13:59:48.723107 26771 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 13:59:48.724066 26771 catalog_manager.cc:1504] Loading token signing keys...
I20240502 13:59:48.734629 26771 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 60bad1ef960c4e09b6fb0fe11be17da2: Generated new TSK 0
I20240502 13:59:48.735380 26771 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 13:59:48.741843 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45215
--builtin_ntp_servers=127.24.66.84:37497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 13:59:48.852924 26777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:48.853113 26777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:48.853155 26777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:48.856096 26777 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:48.856155 26777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:48.856228 26777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20240502 13:59:48.860334 26777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:37497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45215
--never_fsync=true
--heap_profile_path=/tmp/kudu.26777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:48.861184 26777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:48.862238 26777 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:48.870355 26782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:48.870828 26785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:48.871613 26783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:48.872299 26777 server_base.cc:1034] running on GCE node
I20240502 13:59:48.872705 26777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:48.873322 26777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:48.874480 26777 hybrid_clock.cc:648] HybridClock initialized: now 1714658388874446 us; error 46 us; skew 500 ppm
I20240502 13:59:48.876739 26777 webserver.cc:460] Webserver started at http://127.24.66.65:35485/ using document root <none> and password file <none>
I20240502 13:59:48.877388 26777 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:48.877472 26777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:48.877672 26777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:48.879673 26777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "d3e36186faa442b7bd87357e74d47f68"
format_stamp: "Formatted at 2024-05-02 13:59:48 on dist-test-slave-5npg"
I20240502 13:59:48.880277 26777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "d3e36186faa442b7bd87357e74d47f68"
format_stamp: "Formatted at 2024-05-02 13:59:48 on dist-test-slave-5npg"
I20240502 13:59:48.883486 26777 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20240502 13:59:48.885668 26791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:48.886655 26777 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:48.886781 26777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "d3e36186faa442b7bd87357e74d47f68"
format_stamp: "Formatted at 2024-05-02 13:59:48 on dist-test-slave-5npg"
I20240502 13:59:48.886860 26777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:48.913285 26777 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:48.913938 26777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:48.914099 26777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:48.914621 26777 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:48.915488 26777 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:48.915539 26777 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:48.915596 26777 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:48.915627 26777 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:48.923916 26777 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:40841
I20240502 13:59:48.923946 26905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:40841 every 8 connection(s)
I20240502 13:59:48.924806 26777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 13:59:48.927816 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 26777
I20240502 13:59:48.927902 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 13:59:48.929780 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45215
--builtin_ntp_servers=127.24.66.84:37497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 13:59:48.933418 26906 heartbeater.cc:347] Connected to a master server at 127.24.66.126:45215
I20240502 13:59:48.933612 26906 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:48.934072 26906 heartbeater.cc:511] Master 127.24.66.126:45215 requested a full tablet report, sending...
I20240502 13:59:48.935678 26716 ts_manager.cc:194] Registered new tserver with Master: d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841)
I20240502 13:59:48.937386 26716 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.65:33853
W20240502 13:59:49.035910 26909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:49.036121 26909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:49.036163 26909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:49.039101 26909 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:49.039160 26909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:49.039227 26909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20240502 13:59:49.042732 26909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:37497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45215
--never_fsync=true
--heap_profile_path=/tmp/kudu.26909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:49.043529 26909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:49.044732 26909 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:49.052124 26917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:49.052901 26909 server_base.cc:1034] running on GCE node
W20240502 13:59:49.053561 26915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:49.054191 26914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:49.054558 26909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:49.055071 26909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:49.056242 26909 hybrid_clock.cc:648] HybridClock initialized: now 1714658389056217 us; error 36 us; skew 500 ppm
I20240502 13:59:49.058019 26909 webserver.cc:460] Webserver started at http://127.24.66.66:46479/ using document root <none> and password file <none>
I20240502 13:59:49.058526 26909 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:49.058578 26909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:49.058763 26909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:49.060266 26909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "256ce64303ec49e1a5cd8e0ffee436b0"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-5npg"
I20240502 13:59:49.060752 26909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "256ce64303ec49e1a5cd8e0ffee436b0"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-5npg"
I20240502 13:59:49.063758 26909 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 13:59:49.065814 26923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.066710 26909 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:49.066825 26909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "256ce64303ec49e1a5cd8e0ffee436b0"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-5npg"
I20240502 13:59:49.066908 26909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:49.084122 26909 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:49.084721 26909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:49.084865 26909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:49.085417 26909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:49.086294 26909 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:49.086338 26909 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.086374 26909 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:49.086395 26909 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.094477 26909 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:40473
I20240502 13:59:49.094520 27037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:40473 every 8 connection(s)
I20240502 13:59:49.095347 26909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 13:59:49.103246 27038 heartbeater.cc:347] Connected to a master server at 127.24.66.126:45215
I20240502 13:59:49.103466 27038 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:49.103993 27038 heartbeater.cc:511] Master 127.24.66.126:45215 requested a full tablet report, sending...
I20240502 13:59:49.104853 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 26909
I20240502 13:59:49.104948 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 13:59:49.105031 26716 ts_manager.cc:194] Registered new tserver with Master: 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473)
I20240502 13:59:49.105573 26716 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.66:39925
I20240502 13:59:49.106979 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45215
--builtin_ntp_servers=127.24.66.84:37497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 13:59:49.210943 27041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:49.211139 27041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:49.211185 27041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:49.214236 27041 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:49.214294 27041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:49.214376 27041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20240502 13:59:49.217904 27041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:37497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45215
--never_fsync=true
--heap_profile_path=/tmp/kudu.27041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:49.218709 27041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:49.219776 27041 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:49.227726 27047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:49.228406 27049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:49.228952 27046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:49.229691 27041 server_base.cc:1034] running on GCE node
I20240502 13:59:49.230016 27041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:49.230515 27041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:49.231649 27041 hybrid_clock.cc:648] HybridClock initialized: now 1714658389231635 us; error 34 us; skew 500 ppm
I20240502 13:59:49.233327 27041 webserver.cc:460] Webserver started at http://127.24.66.67:45045/ using document root <none> and password file <none>
I20240502 13:59:49.233834 27041 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:49.233914 27041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:49.234094 27041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:49.235623 27041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "0280081e4ff74896832ec3985c372b3a"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-5npg"
I20240502 13:59:49.236056 27041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "0280081e4ff74896832ec3985c372b3a"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-5npg"
I20240502 13:59:49.239143 27041 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20240502 13:59:49.241237 27055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.242233 27041 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:49.242341 27041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "0280081e4ff74896832ec3985c372b3a"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-5npg"
I20240502 13:59:49.242416 27041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:49.260596 27041 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:49.261114 27041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:49.261247 27041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:49.261739 27041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:49.262588 27041 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:49.262645 27041 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.262706 27041 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:49.262737 27041 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.271059 27041 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:35733
I20240502 13:59:49.271106 27169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:35733 every 8 connection(s)
I20240502 13:59:49.271955 27041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 13:59:49.279615 27170 heartbeater.cc:347] Connected to a master server at 127.24.66.126:45215
I20240502 13:59:49.279800 27170 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:49.280261 27170 heartbeater.cc:511] Master 127.24.66.126:45215 requested a full tablet report, sending...
I20240502 13:59:49.281193 26715 ts_manager.cc:194] Registered new tserver with Master: 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733)
I20240502 13:59:49.281749 26715 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.67:57699
I20240502 13:59:49.282212 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 27041
I20240502 13:59:49.282325 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 13:59:49.284106 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.68:0
--local_ip_for_outbound_sockets=127.24.66.68
--webserver_interface=127.24.66.68
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45215
--builtin_ntp_servers=127.24.66.84:37497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 13:59:49.389720 27173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:49.389910 27173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:49.389952 27173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:49.392930 27173 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:49.392984 27173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:49.393046 27173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.68
I20240502 13:59:49.396428 27173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:37497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.24.66.68
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45215
--never_fsync=true
--heap_profile_path=/tmp/kudu.27173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.68
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:49.397152 27173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:49.398165 27173 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:49.406169 27179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:49.406847 27178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:49.406966 27181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:49.407604 27173 server_base.cc:1034] running on GCE node
I20240502 13:59:49.407904 27173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:49.408391 27173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:49.409556 27173 hybrid_clock.cc:648] HybridClock initialized: now 1714658389409542 us; error 26 us; skew 500 ppm
I20240502 13:59:49.411139 27173 webserver.cc:460] Webserver started at http://127.24.66.68:36877/ using document root <none> and password file <none>
I20240502 13:59:49.411623 27173 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:49.411672 27173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:49.411829 27173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:49.413422 27173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "b4fc3c795b1d4505b3a2903532d38b52"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-5npg"
I20240502 13:59:49.413838 27173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "b4fc3c795b1d4505b3a2903532d38b52"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-5npg"
I20240502 13:59:49.416852 27173 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20240502 13:59:49.418865 27187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.419819 27173 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:49.419909 27173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "b4fc3c795b1d4505b3a2903532d38b52"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-5npg"
I20240502 13:59:49.419979 27173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:49.436035 27173 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:49.436654 27173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:49.436785 27173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:49.437238 27173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:49.438073 27173 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:49.438115 27173 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.438153 27173 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:49.438174 27173 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.446303 27173 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.68:35167
I20240502 13:59:49.446352 27300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.68:35167 every 8 connection(s)
I20240502 13:59:49.447176 27173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 13:59:49.449738 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 27173
I20240502 13:59:49.449860 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 13:59:49.451951 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.69:0
--local_ip_for_outbound_sockets=127.24.66.69
--webserver_interface=127.24.66.69
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45215
--builtin_ntp_servers=127.24.66.84:37497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 13:59:49.455938 27301 heartbeater.cc:347] Connected to a master server at 127.24.66.126:45215
I20240502 13:59:49.456192 27301 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:49.456912 27301 heartbeater.cc:511] Master 127.24.66.126:45215 requested a full tablet report, sending...
I20240502 13:59:49.457983 26715 ts_manager.cc:194] Registered new tserver with Master: b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68:35167)
I20240502 13:59:49.458518 26715 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.68:46101
W20240502 13:59:49.556977 27304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:49.557186 27304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:49.557229 27304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:49.560195 27304 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:49.560257 27304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:49.560319 27304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.69
I20240502 13:59:49.564059 27304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:37497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.69:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.24.66.69
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45215
--never_fsync=true
--heap_profile_path=/tmp/kudu.27304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.69
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:49.564950 27304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:49.566005 27304 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:49.573514 27310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:49.574070 27313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:49.574836 27309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:49.575562 27304 server_base.cc:1034] running on GCE node
I20240502 13:59:49.575938 27304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:49.576519 27304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:49.577689 27304 hybrid_clock.cc:648] HybridClock initialized: now 1714658389577680 us; error 29 us; skew 500 ppm
I20240502 13:59:49.579532 27304 webserver.cc:460] Webserver started at http://127.24.66.69:32901/ using document root <none> and password file <none>
I20240502 13:59:49.580062 27304 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:49.580122 27304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:49.580291 27304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:49.581938 27304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "e44f8c16a14a48a38a5a74af2899c217"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-5npg"
I20240502 13:59:49.582386 27304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "e44f8c16a14a48a38a5a74af2899c217"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-5npg"
I20240502 13:59:49.585520 27304 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 13:59:49.587706 27319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.588692 27304 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 13:59:49.588802 27304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "e44f8c16a14a48a38a5a74af2899c217"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-5npg"
I20240502 13:59:49.588898 27304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:49.599812 27304 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:49.600517 27304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:49.600669 27304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:49.601192 27304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:49.602084 27304 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:49.602135 27304 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.602197 27304 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:49.602229 27304 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.610901 27304 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.69:39107
I20240502 13:59:49.610929 27433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.69:39107 every 8 connection(s)
I20240502 13:59:49.611827 27304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 13:59:49.617851 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 27304
I20240502 13:59:49.617977 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 13:59:49.620040 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.70:0
--local_ip_for_outbound_sockets=127.24.66.70
--webserver_interface=127.24.66.70
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45215
--builtin_ntp_servers=127.24.66.84:37497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 13:59:49.620260 27434 heartbeater.cc:347] Connected to a master server at 127.24.66.126:45215
I20240502 13:59:49.620488 27434 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:49.620990 27434 heartbeater.cc:511] Master 127.24.66.126:45215 requested a full tablet report, sending...
I20240502 13:59:49.622041 26715 ts_manager.cc:194] Registered new tserver with Master: e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107)
I20240502 13:59:49.622581 26715 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.69:45005
W20240502 13:59:49.733662 27437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:49.733889 27437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:49.733939 27437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:49.737254 27437 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:49.737346 27437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:49.737481 27437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.70
I20240502 13:59:49.741402 27437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:37497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.70:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.24.66.70
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45215
--never_fsync=true
--heap_profile_path=/tmp/kudu.27437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.70
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:49.742309 27437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:49.743438 27437 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:49.751610 27442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:49.752197 27445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:49.752426 27443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:49.753019 27437 server_base.cc:1034] running on GCE node
I20240502 13:59:49.753355 27437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:49.753876 27437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:49.755025 27437 hybrid_clock.cc:648] HybridClock initialized: now 1714658389755013 us; error 25 us; skew 500 ppm
I20240502 13:59:49.756829 27437 webserver.cc:460] Webserver started at http://127.24.66.70:34597/ using document root <none> and password file <none>
I20240502 13:59:49.757334 27437 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:49.757383 27437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:49.757555 27437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:49.759117 27437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "34b161bd719d4d72baf8da4f40aef136"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-5npg"
I20240502 13:59:49.759524 27437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "34b161bd719d4d72baf8da4f40aef136"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-5npg"
I20240502 13:59:49.762673 27437 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20240502 13:59:49.764853 27451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.765798 27437 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 13:59:49.765893 27437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "34b161bd719d4d72baf8da4f40aef136"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-5npg"
I20240502 13:59:49.765986 27437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:49.812426 27437 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:49.813074 27437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:49.813225 27437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:49.813824 27437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:49.814699 27437 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:49.814746 27437 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.814798 27437 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:49.814828 27437 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.823154 27437 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.70:38217
I20240502 13:59:49.823253 27565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.70:38217 every 8 connection(s)
I20240502 13:59:49.824133 27437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 13:59:49.827291 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 27437
I20240502 13:59:49.827414 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240502 13:59:49.829641 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.71:0
--local_ip_for_outbound_sockets=127.24.66.71
--webserver_interface=127.24.66.71
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45215
--builtin_ntp_servers=127.24.66.84:37497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 13:59:49.833330 27566 heartbeater.cc:347] Connected to a master server at 127.24.66.126:45215
I20240502 13:59:49.833556 27566 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:49.834095 27566 heartbeater.cc:511] Master 127.24.66.126:45215 requested a full tablet report, sending...
I20240502 13:59:49.835091 26715 ts_manager.cc:194] Registered new tserver with Master: 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217)
I20240502 13:59:49.835628 26715 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.70:48237
W20240502 13:59:49.936939 27569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:49.937163 27569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:49.937234 27569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20240502 13:59:49.939507 26906 heartbeater.cc:503] Master 127.24.66.126:45215 was elected leader, sending a full tablet report...
W20240502 13:59:49.940241 27569 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:49.940307 27569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:49.940447 27569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.71
I20240502 13:59:49.943985 27569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:37497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.71:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.24.66.71
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45215
--never_fsync=true
--heap_profile_path=/tmp/kudu.27569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.71
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:49.944866 27569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:49.945974 27569 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:49.953656 27577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:49.954386 27574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:49.955101 27575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:49.955716 27569 server_base.cc:1034] running on GCE node
I20240502 13:59:49.956060 27569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:49.956630 27569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:49.957773 27569 hybrid_clock.cc:648] HybridClock initialized: now 1714658389957752 us; error 37 us; skew 500 ppm
I20240502 13:59:49.959432 27569 webserver.cc:460] Webserver started at http://127.24.66.71:46589/ using document root <none> and password file <none>
I20240502 13:59:49.959973 27569 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:49.960029 27569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:49.960211 27569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 13:59:49.961864 27569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "5f9ddfa1f6fc484d8fb96075a8efb649"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-5npg"
I20240502 13:59:49.962301 27569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "5f9ddfa1f6fc484d8fb96075a8efb649"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-5npg"
I20240502 13:59:49.965453 27569 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 13:59:49.967507 27583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.968498 27569 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 13:59:49.968602 27569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "5f9ddfa1f6fc484d8fb96075a8efb649"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-5npg"
I20240502 13:59:49.968686 27569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:49.988768 27569 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:49.989394 27569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:49.989562 27569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:49.990140 27569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:49.991037 27569 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 13:59:49.991096 27569 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.991159 27569 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 13:59:49.991192 27569 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:49.999251 27569 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.71:42201
I20240502 13:59:49.999300 27698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.71:42201 every 8 connection(s)
I20240502 13:59:50.000164 27569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 13:59:50.005736 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 27569
I20240502 13:59:50.005852 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240502 13:59:50.008478 27699 heartbeater.cc:347] Connected to a master server at 127.24.66.126:45215
I20240502 13:59:50.008678 27699 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:50.009150 27699 heartbeater.cc:511] Master 127.24.66.126:45215 requested a full tablet report, sending...
I20240502 13:59:50.010003 26715 ts_manager.cc:194] Registered new tserver with Master: 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71:42201)
I20240502 13:59:50.010484 26715 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.71:41079
I20240502 13:59:50.019400 24841 external_mini_cluster.cc:930] 7 TS(s) registered with all masters
I20240502 13:59:50.027433 26715 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50400:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240502 13:59:50.045848 27367 tablet_service.cc:1467] Processing CreateTablet for tablet 4a156115301a4d40bfc7f2bd11f135bf (DEFAULT_TABLE table=TestTable [id=beb77348bd534888844c42de955fc55b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:50.045843 27499 tablet_service.cc:1467] Processing CreateTablet for tablet 4a156115301a4d40bfc7f2bd11f135bf (DEFAULT_TABLE table=TestTable [id=beb77348bd534888844c42de955fc55b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:50.045845 27235 tablet_service.cc:1467] Processing CreateTablet for tablet 4a156115301a4d40bfc7f2bd11f135bf (DEFAULT_TABLE table=TestTable [id=beb77348bd534888844c42de955fc55b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:50.046797 27367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a156115301a4d40bfc7f2bd11f135bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:50.046797 27499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a156115301a4d40bfc7f2bd11f135bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:50.046799 27235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a156115301a4d40bfc7f2bd11f135bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:50.047355 26971 tablet_service.cc:1467] Processing CreateTablet for tablet 4a156115301a4d40bfc7f2bd11f135bf (DEFAULT_TABLE table=TestTable [id=beb77348bd534888844c42de955fc55b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:50.048084 26971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a156115301a4d40bfc7f2bd11f135bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:50.053715 27722 tablet_bootstrap.cc:492] T 4a156115301a4d40bfc7f2bd11f135bf P b4fc3c795b1d4505b3a2903532d38b52: Bootstrap starting.
I20240502 13:59:50.054112 27725 tablet_bootstrap.cc:492] T 4a156115301a4d40bfc7f2bd11f135bf P 256ce64303ec49e1a5cd8e0ffee436b0: Bootstrap starting.
I20240502 13:59:50.055166 27723 tablet_bootstrap.cc:492] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217: Bootstrap starting.
I20240502 13:59:50.055572 27724 tablet_bootstrap.cc:492] T 4a156115301a4d40bfc7f2bd11f135bf P 34b161bd719d4d72baf8da4f40aef136: Bootstrap starting.
I20240502 13:59:50.055881 27725 tablet_bootstrap.cc:654] T 4a156115301a4d40bfc7f2bd11f135bf P 256ce64303ec49e1a5cd8e0ffee436b0: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:50.055881 27722 tablet_bootstrap.cc:654] T 4a156115301a4d40bfc7f2bd11f135bf P b4fc3c795b1d4505b3a2903532d38b52: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:50.056730 27722 log.cc:824] T 4a156115301a4d40bfc7f2bd11f135bf P b4fc3c795b1d4505b3a2903532d38b52: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:50.056730 27725 log.cc:824] T 4a156115301a4d40bfc7f2bd11f135bf P 256ce64303ec49e1a5cd8e0ffee436b0: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:50.057637 27724 tablet_bootstrap.cc:654] T 4a156115301a4d40bfc7f2bd11f135bf P 34b161bd719d4d72baf8da4f40aef136: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:50.058709 27725 tablet_bootstrap.cc:492] T 4a156115301a4d40bfc7f2bd11f135bf P 256ce64303ec49e1a5cd8e0ffee436b0: No bootstrap required, opened a new log
I20240502 13:59:50.058709 27722 tablet_bootstrap.cc:492] T 4a156115301a4d40bfc7f2bd11f135bf P b4fc3c795b1d4505b3a2903532d38b52: No bootstrap required, opened a new log
I20240502 13:59:50.058883 27725 ts_tablet_manager.cc:1367] T 4a156115301a4d40bfc7f2bd11f135bf P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240502 13:59:50.058883 27722 ts_tablet_manager.cc:1367] T 4a156115301a4d40bfc7f2bd11f135bf P b4fc3c795b1d4505b3a2903532d38b52: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 13:59:50.059790 27632 tablet_service.cc:1467] Processing CreateTablet for tablet 4a156115301a4d40bfc7f2bd11f135bf (DEFAULT_TABLE table=TestTable [id=beb77348bd534888844c42de955fc55b]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:50.060540 27632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a156115301a4d40bfc7f2bd11f135bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:50.061993 27725 raft_consensus.cc:348] T 4a156115301a4d40bfc7f2bd11f135bf P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:50.061993 27722 raft_consensus.cc:348] T 4a156115301a4d40bfc7f2bd11f135bf P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:50.062211 27722 raft_consensus.cc:374] T 4a156115301a4d40bfc7f2bd11f135bf P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:50.062251 27725 raft_consensus.cc:374] T 4a156115301a4d40bfc7f2bd11f135bf P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:50.062290 27722 raft_consensus.cc:729] T 4a156115301a4d40bfc7f2bd11f135bf P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4fc3c795b1d4505b3a2903532d38b52, State: Initialized, Role: FOLLOWER
I20240502 13:59:50.062320 27725 raft_consensus.cc:729] T 4a156115301a4d40bfc7f2bd11f135bf P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 256ce64303ec49e1a5cd8e0ffee436b0, State: Initialized, Role: FOLLOWER
I20240502 13:59:50.062769 27725 consensus_queue.cc:260] T 4a156115301a4d40bfc7f2bd11f135bf P 256ce64303ec49e1a5cd8e0ffee436b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:50.062767 27722 consensus_queue.cc:260] T 4a156115301a4d40bfc7f2bd11f135bf P b4fc3c795b1d4505b3a2903532d38b52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:50.063602 27725 ts_tablet_manager.cc:1398] T 4a156115301a4d40bfc7f2bd11f135bf P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240502 13:59:50.063602 27722 ts_tablet_manager.cc:1398] T 4a156115301a4d40bfc7f2bd11f135bf P b4fc3c795b1d4505b3a2903532d38b52: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240502 13:59:50.063966 27038 heartbeater.cc:503] Master 127.24.66.126:45215 was elected leader, sending a full tablet report...
I20240502 13:59:50.064016 27301 heartbeater.cc:503] Master 127.24.66.126:45215 was elected leader, sending a full tablet report...
I20240502 13:59:50.068096 27730 tablet_bootstrap.cc:492] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap starting.
I20240502 13:59:50.068805 27723 tablet_bootstrap.cc:654] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:50.069044 27724 log.cc:824] T 4a156115301a4d40bfc7f2bd11f135bf P 34b161bd719d4d72baf8da4f40aef136: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:50.069608 27723 log.cc:824] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:50.070106 27730 tablet_bootstrap.cc:654] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:50.070758 27724 tablet_bootstrap.cc:492] T 4a156115301a4d40bfc7f2bd11f135bf P 34b161bd719d4d72baf8da4f40aef136: No bootstrap required, opened a new log
I20240502 13:59:50.070864 27730 log.cc:824] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:50.070957 27724 ts_tablet_manager.cc:1367] T 4a156115301a4d40bfc7f2bd11f135bf P 34b161bd719d4d72baf8da4f40aef136: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.000s
I20240502 13:59:50.071215 27723 tablet_bootstrap.cc:492] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217: No bootstrap required, opened a new log
I20240502 13:59:50.071373 27723 ts_tablet_manager.cc:1367] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.000s
I20240502 13:59:50.072409 27730 tablet_bootstrap.cc:492] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649: No bootstrap required, opened a new log
I20240502 13:59:50.072587 27730 ts_tablet_manager.cc:1367] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 13:59:50.074049 27724 raft_consensus.cc:348] T 4a156115301a4d40bfc7f2bd11f135bf P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:50.074529 27724 raft_consensus.cc:374] T 4a156115301a4d40bfc7f2bd11f135bf P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:50.074630 27724 raft_consensus.cc:729] T 4a156115301a4d40bfc7f2bd11f135bf P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34b161bd719d4d72baf8da4f40aef136, State: Initialized, Role: FOLLOWER
I20240502 13:59:50.075064 27724 consensus_queue.cc:260] T 4a156115301a4d40bfc7f2bd11f135bf P 34b161bd719d4d72baf8da4f40aef136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:50.075001 27723 raft_consensus.cc:348] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:50.075187 27723 raft_consensus.cc:374] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:50.075300 27723 raft_consensus.cc:729] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44f8c16a14a48a38a5a74af2899c217, State: Initialized, Role: FOLLOWER
I20240502 13:59:50.075635 27566 heartbeater.cc:503] Master 127.24.66.126:45215 was elected leader, sending a full tablet report...
I20240502 13:59:50.075860 27724 ts_tablet_manager.cc:1398] T 4a156115301a4d40bfc7f2bd11f135bf P 34b161bd719d4d72baf8da4f40aef136: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240502 13:59:50.075917 27723 consensus_queue.cc:260] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:50.076588 27434 heartbeater.cc:503] Master 127.24.66.126:45215 was elected leader, sending a full tablet report...
I20240502 13:59:50.076994 27723 ts_tablet_manager.cc:1398] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20240502 13:59:50.077288 27730 raft_consensus.cc:348] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:50.077498 27730 raft_consensus.cc:374] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:50.077600 27730 raft_consensus.cc:729] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9ddfa1f6fc484d8fb96075a8efb649, State: Initialized, Role: FOLLOWER
I20240502 13:59:50.078003 27730 consensus_queue.cc:260] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:50.078903 27730 ts_tablet_manager.cc:1398] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240502 13:59:50.079201 27699 heartbeater.cc:503] Master 127.24.66.126:45215 was elected leader, sending a full tablet report...
W20240502 13:59:50.096729 27039 tablet.cc:2369] T 4a156115301a4d40bfc7f2bd11f135bf P 256ce64303ec49e1a5cd8e0ffee436b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 13:59:50.112852 27435 tablet.cc:2369] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 13:59:50.180415 27735 raft_consensus.cc:482] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:50.180616 27735 raft_consensus.cc:504] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:50.181778 27735 leader_election.cc:288] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68:35167), 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217), 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473), 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71:42201)
I20240502 13:59:50.187194 26991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a156115301a4d40bfc7f2bd11f135bf" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" is_pre_election: true
I20240502 13:59:50.187198 27519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a156115301a4d40bfc7f2bd11f135bf" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b161bd719d4d72baf8da4f40aef136" is_pre_election: true
I20240502 13:59:50.187531 26991 raft_consensus.cc:2454] T 4a156115301a4d40bfc7f2bd11f135bf P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 0.
I20240502 13:59:50.187544 27519 raft_consensus.cc:2454] T 4a156115301a4d40bfc7f2bd11f135bf P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 0.
I20240502 13:59:50.187680 27255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a156115301a4d40bfc7f2bd11f135bf" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4fc3c795b1d4505b3a2903532d38b52" is_pre_election: true
I20240502 13:59:50.187974 27255 raft_consensus.cc:2454] T 4a156115301a4d40bfc7f2bd11f135bf P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 0.
I20240502 13:59:50.188001 27653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a156115301a4d40bfc7f2bd11f135bf" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" is_pre_election: true
I20240502 13:59:50.188048 27320 leader_election.cc:302] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 256ce64303ec49e1a5cd8e0ffee436b0, 34b161bd719d4d72baf8da4f40aef136, e44f8c16a14a48a38a5a74af2899c217; no voters:
I20240502 13:59:50.188239 27653 raft_consensus.cc:2454] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 0.
I20240502 13:59:50.188374 27735 raft_consensus.cc:2781] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:50.188457 27735 raft_consensus.cc:482] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:50.188513 27735 raft_consensus.cc:3037] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:50.189488 27735 raft_consensus.cc:504] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:50.189947 27735 leader_election.cc:288] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [CANDIDATE]: Term 1 election: Requested vote from peers b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68:35167), 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217), 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473), 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71:42201)
I20240502 13:59:50.190222 27255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a156115301a4d40bfc7f2bd11f135bf" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4fc3c795b1d4505b3a2903532d38b52"
I20240502 13:59:50.190225 27519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a156115301a4d40bfc7f2bd11f135bf" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b161bd719d4d72baf8da4f40aef136"
I20240502 13:59:50.190339 27653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a156115301a4d40bfc7f2bd11f135bf" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649"
I20240502 13:59:50.190361 27519 raft_consensus.cc:3037] T 4a156115301a4d40bfc7f2bd11f135bf P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:50.190361 27255 raft_consensus.cc:3037] T 4a156115301a4d40bfc7f2bd11f135bf P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:50.190362 26991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a156115301a4d40bfc7f2bd11f135bf" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0"
I20240502 13:59:50.190444 27653 raft_consensus.cc:3037] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:50.190476 26991 raft_consensus.cc:3037] T 4a156115301a4d40bfc7f2bd11f135bf P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:50.191429 27653 raft_consensus.cc:2454] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 1.
I20240502 13:59:50.191429 27519 raft_consensus.cc:2454] T 4a156115301a4d40bfc7f2bd11f135bf P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 1.
I20240502 13:59:50.191447 27255 raft_consensus.cc:2454] T 4a156115301a4d40bfc7f2bd11f135bf P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 1.
I20240502 13:59:50.191637 26991 raft_consensus.cc:2454] T 4a156115301a4d40bfc7f2bd11f135bf P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 1.
I20240502 13:59:50.191793 27320 leader_election.cc:302] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 5f9ddfa1f6fc484d8fb96075a8efb649, b4fc3c795b1d4505b3a2903532d38b52, e44f8c16a14a48a38a5a74af2899c217; no voters:
I20240502 13:59:50.191990 27735 raft_consensus.cc:2781] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:50.192209 27735 raft_consensus.cc:686] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [term 1 LEADER]: Becoming Leader. State: Replica: e44f8c16a14a48a38a5a74af2899c217, State: Running, Role: LEADER
I20240502 13:59:50.192545 27735 consensus_queue.cc:237] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:50.195683 26714 catalog_manager.cc:5484] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 reported cstate change: term changed from 0 to 1, leader changed from <none> to e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69). New cstate: current_term: 1 leader_uuid: "e44f8c16a14a48a38a5a74af2899c217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } }
W20240502 13:59:50.198364 27302 tablet.cc:2369] T 4a156115301a4d40bfc7f2bd11f135bf P b4fc3c795b1d4505b3a2903532d38b52: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 13:59:50.234120 24841 external_mini_cluster.cc:930] 7 TS(s) registered with all masters
I20240502 13:59:50.235504 24841 ts_itest-base.cc:233] Waiting for 0 tablets on tserver d3e36186faa442b7bd87357e74d47f68 to finish bootstrapping
I20240502 13:59:50.241065 24841 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 256ce64303ec49e1a5cd8e0ffee436b0 to finish bootstrapping
I20240502 13:59:50.248780 24841 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 0280081e4ff74896832ec3985c372b3a to finish bootstrapping
W20240502 13:59:50.250983 27700 tablet.cc:2369] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 13:59:50.253952 24841 ts_itest-base.cc:233] Waiting for 1 tablets on tserver b4fc3c795b1d4505b3a2903532d38b52 to finish bootstrapping
I20240502 13:59:50.257915 24841 ts_itest-base.cc:233] Waiting for 1 tablets on tserver e44f8c16a14a48a38a5a74af2899c217 to finish bootstrapping
I20240502 13:59:50.262144 24841 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 34b161bd719d4d72baf8da4f40aef136 to finish bootstrapping
I20240502 13:59:50.266184 24841 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 5f9ddfa1f6fc484d8fb96075a8efb649 to finish bootstrapping
I20240502 13:59:50.270213 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 27569
I20240502 13:59:50.283668 27170 heartbeater.cc:503] Master 127.24.66.126:45215 was elected leader, sending a full tablet report...
W20240502 13:59:50.325109 27567 tablet.cc:2369] T 4a156115301a4d40bfc7f2bd11f135bf P 34b161bd719d4d72baf8da4f40aef136: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 13:59:50.589089 27735 consensus_queue.cc:1035] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:50.597769 27735 consensus_queue.cc:1035] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 13:59:50.602221 27320 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.66.71:42201: connect: Connection refused (error 111)
I20240502 13:59:50.603597 27751 consensus_queue.cc:1035] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 13:59:50.603879 27320 consensus_peers.cc:485] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 -> Peer 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71:42201): Couldn't send request to peer 5f9ddfa1f6fc484d8fb96075a8efb649. Status: Network error: Client connection negotiation failed: client connection to 127.24.66.71:42201: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 13:59:53.275861 27320 consensus_peers.cc:485] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 -> Peer 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71:42201): Couldn't send request to peer 5f9ddfa1f6fc484d8fb96075a8efb649. Status: Network error: Client connection negotiation failed: client connection to 127.24.66.71:42201: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 13:59:54.027838 26716 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50400:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 13:59:54.035334 27235 tablet_service.cc:1467] Processing CreateTablet for tablet f4bb31e5c0454e54a7b3855445120783 (DEFAULT_TABLE table=rebalance_test_table_0 [id=360c53214ba44103b4436952fc42eddc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.035333 26971 tablet_service.cc:1467] Processing CreateTablet for tablet f4bb31e5c0454e54a7b3855445120783 (DEFAULT_TABLE table=rebalance_test_table_0 [id=360c53214ba44103b4436952fc42eddc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.035663 27235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4bb31e5c0454e54a7b3855445120783. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.035663 26971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4bb31e5c0454e54a7b3855445120783. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.037828 27498 tablet_service.cc:1467] Processing CreateTablet for tablet abb00d05d9704f5e8a78cb9fd1866e2a (DEFAULT_TABLE table=rebalance_test_table_0 [id=360c53214ba44103b4436952fc42eddc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.038161 27498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abb00d05d9704f5e8a78cb9fd1866e2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.038198 27499 tablet_service.cc:1467] Processing CreateTablet for tablet 53a68b75cbce41d69432f21f2cfbf256 (DEFAULT_TABLE table=rebalance_test_table_0 [id=360c53214ba44103b4436952fc42eddc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.038421 27499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53a68b75cbce41d69432f21f2cfbf256. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.038456 26970 tablet_service.cc:1467] Processing CreateTablet for tablet 53a68b75cbce41d69432f21f2cfbf256 (DEFAULT_TABLE table=rebalance_test_table_0 [id=360c53214ba44103b4436952fc42eddc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.038713 26970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53a68b75cbce41d69432f21f2cfbf256. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.040943 27366 tablet_service.cc:1467] Processing CreateTablet for tablet abb00d05d9704f5e8a78cb9fd1866e2a (DEFAULT_TABLE table=rebalance_test_table_0 [id=360c53214ba44103b4436952fc42eddc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.041267 27366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abb00d05d9704f5e8a78cb9fd1866e2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.041496 27777 tablet_bootstrap.cc:492] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136: Bootstrap starting.
I20240502 13:59:54.041939 27234 tablet_service.cc:1467] Processing CreateTablet for tablet 53a68b75cbce41d69432f21f2cfbf256 (DEFAULT_TABLE table=rebalance_test_table_0 [id=360c53214ba44103b4436952fc42eddc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.042214 27234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53a68b75cbce41d69432f21f2cfbf256. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.043838 27778 tablet_bootstrap.cc:492] T abb00d05d9704f5e8a78cb9fd1866e2a P e44f8c16a14a48a38a5a74af2899c217: Bootstrap starting.
I20240502 13:59:54.044278 27777 tablet_bootstrap.cc:654] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.044621 26969 tablet_service.cc:1467] Processing CreateTablet for tablet abb00d05d9704f5e8a78cb9fd1866e2a (DEFAULT_TABLE table=rebalance_test_table_0 [id=360c53214ba44103b4436952fc42eddc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.044903 26969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abb00d05d9704f5e8a78cb9fd1866e2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.045018 27778 tablet_bootstrap.cc:654] T abb00d05d9704f5e8a78cb9fd1866e2a P e44f8c16a14a48a38a5a74af2899c217: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.046187 27103 tablet_service.cc:1467] Processing CreateTablet for tablet f4bb31e5c0454e54a7b3855445120783 (DEFAULT_TABLE table=rebalance_test_table_0 [id=360c53214ba44103b4436952fc42eddc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.045950 27102 tablet_service.cc:1467] Processing CreateTablet for tablet 53a68b75cbce41d69432f21f2cfbf256 (DEFAULT_TABLE table=rebalance_test_table_0 [id=360c53214ba44103b4436952fc42eddc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.040943 27367 tablet_service.cc:1467] Processing CreateTablet for tablet f4bb31e5c0454e54a7b3855445120783 (DEFAULT_TABLE table=rebalance_test_table_0 [id=360c53214ba44103b4436952fc42eddc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.046965 27102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53a68b75cbce41d69432f21f2cfbf256. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.047114 27367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4bb31e5c0454e54a7b3855445120783. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.047313 27103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4bb31e5c0454e54a7b3855445120783. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.045938 27101 tablet_service.cc:1467] Processing CreateTablet for tablet abb00d05d9704f5e8a78cb9fd1866e2a (DEFAULT_TABLE table=rebalance_test_table_0 [id=360c53214ba44103b4436952fc42eddc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.048128 27101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abb00d05d9704f5e8a78cb9fd1866e2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.049103 27777 tablet_bootstrap.cc:492] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136: No bootstrap required, opened a new log
I20240502 13:59:54.049188 27777 ts_tablet_manager.cc:1367] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 13:59:54.049773 27777 raft_consensus.cc:348] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.049907 27777 raft_consensus.cc:374] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.049940 27777 raft_consensus.cc:729] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34b161bd719d4d72baf8da4f40aef136, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.050017 26837 tablet_service.cc:1467] Processing CreateTablet for tablet abb00d05d9704f5e8a78cb9fd1866e2a (DEFAULT_TABLE table=rebalance_test_table_0 [id=360c53214ba44103b4436952fc42eddc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.050333 27777 consensus_queue.cc:260] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.050611 26838 tablet_service.cc:1467] Processing CreateTablet for tablet 53a68b75cbce41d69432f21f2cfbf256 (DEFAULT_TABLE table=rebalance_test_table_0 [id=360c53214ba44103b4436952fc42eddc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.050683 27777 ts_tablet_manager.cc:1398] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:54.050925 26837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abb00d05d9704f5e8a78cb9fd1866e2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.050135 26839 tablet_service.cc:1467] Processing CreateTablet for tablet f4bb31e5c0454e54a7b3855445120783 (DEFAULT_TABLE table=rebalance_test_table_0 [id=360c53214ba44103b4436952fc42eddc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.051223 26839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4bb31e5c0454e54a7b3855445120783. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.052109 26838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53a68b75cbce41d69432f21f2cfbf256. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.055056 27783 tablet_bootstrap.cc:492] T abb00d05d9704f5e8a78cb9fd1866e2a P 0280081e4ff74896832ec3985c372b3a: Bootstrap starting.
I20240502 13:59:54.055437 27777 tablet_bootstrap.cc:492] T 53a68b75cbce41d69432f21f2cfbf256 P 34b161bd719d4d72baf8da4f40aef136: Bootstrap starting.
I20240502 13:59:54.057134 27777 tablet_bootstrap.cc:654] T 53a68b75cbce41d69432f21f2cfbf256 P 34b161bd719d4d72baf8da4f40aef136: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.057593 27783 tablet_bootstrap.cc:654] T abb00d05d9704f5e8a78cb9fd1866e2a P 0280081e4ff74896832ec3985c372b3a: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.057674 27778 tablet_bootstrap.cc:492] T abb00d05d9704f5e8a78cb9fd1866e2a P e44f8c16a14a48a38a5a74af2899c217: No bootstrap required, opened a new log
I20240502 13:59:54.057793 27778 ts_tablet_manager.cc:1367] T abb00d05d9704f5e8a78cb9fd1866e2a P e44f8c16a14a48a38a5a74af2899c217: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 13:59:54.058233 27784 tablet_bootstrap.cc:492] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68: Bootstrap starting.
I20240502 13:59:54.058387 27783 log.cc:824] T abb00d05d9704f5e8a78cb9fd1866e2a P 0280081e4ff74896832ec3985c372b3a: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:54.058432 27778 raft_consensus.cc:348] T abb00d05d9704f5e8a78cb9fd1866e2a P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.058611 27778 raft_consensus.cc:374] T abb00d05d9704f5e8a78cb9fd1866e2a P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.058677 27778 raft_consensus.cc:729] T abb00d05d9704f5e8a78cb9fd1866e2a P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44f8c16a14a48a38a5a74af2899c217, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.058892 27778 consensus_queue.cc:260] T abb00d05d9704f5e8a78cb9fd1866e2a P e44f8c16a14a48a38a5a74af2899c217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.058995 27774 tablet_bootstrap.cc:492] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52: Bootstrap starting.
I20240502 13:59:54.059222 27778 ts_tablet_manager.cc:1398] T abb00d05d9704f5e8a78cb9fd1866e2a P e44f8c16a14a48a38a5a74af2899c217: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:54.059298 27777 tablet_bootstrap.cc:492] T 53a68b75cbce41d69432f21f2cfbf256 P 34b161bd719d4d72baf8da4f40aef136: No bootstrap required, opened a new log
I20240502 13:59:54.059394 27777 ts_tablet_manager.cc:1367] T 53a68b75cbce41d69432f21f2cfbf256 P 34b161bd719d4d72baf8da4f40aef136: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240502 13:59:54.059424 27778 tablet_bootstrap.cc:492] T f4bb31e5c0454e54a7b3855445120783 P e44f8c16a14a48a38a5a74af2899c217: Bootstrap starting.
I20240502 13:59:54.059839 27777 raft_consensus.cc:348] T 53a68b75cbce41d69432f21f2cfbf256 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:54.059965 27777 raft_consensus.cc:374] T 53a68b75cbce41d69432f21f2cfbf256 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.060003 27774 tablet_bootstrap.cc:654] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.060011 27777 raft_consensus.cc:729] T 53a68b75cbce41d69432f21f2cfbf256 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34b161bd719d4d72baf8da4f40aef136, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.060158 27777 consensus_queue.cc:260] T 53a68b75cbce41d69432f21f2cfbf256 P 34b161bd719d4d72baf8da4f40aef136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:54.060494 27778 tablet_bootstrap.cc:654] T f4bb31e5c0454e54a7b3855445120783 P e44f8c16a14a48a38a5a74af2899c217: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.060478 27777 ts_tablet_manager.cc:1398] T 53a68b75cbce41d69432f21f2cfbf256 P 34b161bd719d4d72baf8da4f40aef136: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 13:59:54.061731 27783 tablet_bootstrap.cc:492] T abb00d05d9704f5e8a78cb9fd1866e2a P 0280081e4ff74896832ec3985c372b3a: No bootstrap required, opened a new log
I20240502 13:59:54.061794 27784 tablet_bootstrap.cc:654] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.061911 27783 ts_tablet_manager.cc:1367] T abb00d05d9704f5e8a78cb9fd1866e2a P 0280081e4ff74896832ec3985c372b3a: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20240502 13:59:54.062922 27784 log.cc:824] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:54.063593 27775 tablet_bootstrap.cc:492] T f4bb31e5c0454e54a7b3855445120783 P 256ce64303ec49e1a5cd8e0ffee436b0: Bootstrap starting.
I20240502 13:59:54.063848 27774 tablet_bootstrap.cc:492] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52: No bootstrap required, opened a new log
I20240502 13:59:54.063954 27774 ts_tablet_manager.cc:1367] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 13:59:54.064517 27774 raft_consensus.cc:348] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:54.064641 27774 raft_consensus.cc:374] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.064677 27774 raft_consensus.cc:729] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4fc3c795b1d4505b3a2903532d38b52, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.064831 27774 consensus_queue.cc:260] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:54.065023 27778 tablet_bootstrap.cc:492] T f4bb31e5c0454e54a7b3855445120783 P e44f8c16a14a48a38a5a74af2899c217: No bootstrap required, opened a new log
I20240502 13:59:54.065093 27784 tablet_bootstrap.cc:492] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68: No bootstrap required, opened a new log
I20240502 13:59:54.065169 27774 ts_tablet_manager.cc:1398] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:54.065268 27784 ts_tablet_manager.cc:1367] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20240502 13:59:54.065343 27774 tablet_bootstrap.cc:492] T 53a68b75cbce41d69432f21f2cfbf256 P b4fc3c795b1d4505b3a2903532d38b52: Bootstrap starting.
I20240502 13:59:54.065573 27783 raft_consensus.cc:348] T abb00d05d9704f5e8a78cb9fd1866e2a P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.065809 27783 raft_consensus.cc:374] T abb00d05d9704f5e8a78cb9fd1866e2a P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.065871 27783 raft_consensus.cc:729] T abb00d05d9704f5e8a78cb9fd1866e2a P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0280081e4ff74896832ec3985c372b3a, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.066220 27774 tablet_bootstrap.cc:654] T 53a68b75cbce41d69432f21f2cfbf256 P b4fc3c795b1d4505b3a2903532d38b52: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.066362 27783 consensus_queue.cc:260] T abb00d05d9704f5e8a78cb9fd1866e2a P 0280081e4ff74896832ec3985c372b3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.065182 27778 ts_tablet_manager.cc:1367] T f4bb31e5c0454e54a7b3855445120783 P e44f8c16a14a48a38a5a74af2899c217: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 13:59:54.067237 27783 ts_tablet_manager.cc:1398] T abb00d05d9704f5e8a78cb9fd1866e2a P 0280081e4ff74896832ec3985c372b3a: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20240502 13:59:54.067373 27778 raft_consensus.cc:348] T f4bb31e5c0454e54a7b3855445120783 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:54.067610 27783 tablet_bootstrap.cc:492] T f4bb31e5c0454e54a7b3855445120783 P 0280081e4ff74896832ec3985c372b3a: Bootstrap starting.
I20240502 13:59:54.067625 27778 raft_consensus.cc:374] T f4bb31e5c0454e54a7b3855445120783 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.067751 27778 raft_consensus.cc:729] T f4bb31e5c0454e54a7b3855445120783 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44f8c16a14a48a38a5a74af2899c217, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.067925 27778 consensus_queue.cc:260] T f4bb31e5c0454e54a7b3855445120783 P e44f8c16a14a48a38a5a74af2899c217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:54.068250 27778 ts_tablet_manager.cc:1398] T f4bb31e5c0454e54a7b3855445120783 P e44f8c16a14a48a38a5a74af2899c217: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:54.068693 27775 tablet_bootstrap.cc:654] T f4bb31e5c0454e54a7b3855445120783 P 256ce64303ec49e1a5cd8e0ffee436b0: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.069952 27774 tablet_bootstrap.cc:492] T 53a68b75cbce41d69432f21f2cfbf256 P b4fc3c795b1d4505b3a2903532d38b52: No bootstrap required, opened a new log
I20240502 13:59:54.070086 27774 ts_tablet_manager.cc:1367] T 53a68b75cbce41d69432f21f2cfbf256 P b4fc3c795b1d4505b3a2903532d38b52: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 13:59:54.070498 27784 raft_consensus.cc:348] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:54.070789 27784 raft_consensus.cc:374] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.070865 27784 raft_consensus.cc:729] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3e36186faa442b7bd87357e74d47f68, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.071219 27774 raft_consensus.cc:348] T 53a68b75cbce41d69432f21f2cfbf256 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:54.071383 27774 raft_consensus.cc:374] T 53a68b75cbce41d69432f21f2cfbf256 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.071343 27784 consensus_queue.cc:260] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:54.071435 27783 tablet_bootstrap.cc:654] T f4bb31e5c0454e54a7b3855445120783 P 0280081e4ff74896832ec3985c372b3a: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.071444 27774 raft_consensus.cc:729] T 53a68b75cbce41d69432f21f2cfbf256 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4fc3c795b1d4505b3a2903532d38b52, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.071561 27774 consensus_queue.cc:260] T 53a68b75cbce41d69432f21f2cfbf256 P b4fc3c795b1d4505b3a2903532d38b52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:54.072216 27784 ts_tablet_manager.cc:1398] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20240502 13:59:54.072482 27784 tablet_bootstrap.cc:492] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68: Bootstrap starting.
I20240502 13:59:54.072568 27783 tablet_bootstrap.cc:492] T f4bb31e5c0454e54a7b3855445120783 P 0280081e4ff74896832ec3985c372b3a: No bootstrap required, opened a new log
I20240502 13:59:54.072645 27783 ts_tablet_manager.cc:1367] T f4bb31e5c0454e54a7b3855445120783 P 0280081e4ff74896832ec3985c372b3a: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 13:59:54.071861 27774 ts_tablet_manager.cc:1398] T 53a68b75cbce41d69432f21f2cfbf256 P b4fc3c795b1d4505b3a2903532d38b52: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:54.073156 27783 raft_consensus.cc:348] T f4bb31e5c0454e54a7b3855445120783 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:54.073261 27783 raft_consensus.cc:374] T f4bb31e5c0454e54a7b3855445120783 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.073292 27783 raft_consensus.cc:729] T f4bb31e5c0454e54a7b3855445120783 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0280081e4ff74896832ec3985c372b3a, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.073426 27784 tablet_bootstrap.cc:654] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.073411 27783 consensus_queue.cc:260] T f4bb31e5c0454e54a7b3855445120783 P 0280081e4ff74896832ec3985c372b3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:54.073676 27783 ts_tablet_manager.cc:1398] T f4bb31e5c0454e54a7b3855445120783 P 0280081e4ff74896832ec3985c372b3a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 13:59:54.073832 27783 tablet_bootstrap.cc:492] T 53a68b75cbce41d69432f21f2cfbf256 P 0280081e4ff74896832ec3985c372b3a: Bootstrap starting.
I20240502 13:59:54.074671 27782 raft_consensus.cc:482] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:54.074751 27783 tablet_bootstrap.cc:654] T 53a68b75cbce41d69432f21f2cfbf256 P 0280081e4ff74896832ec3985c372b3a: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.074767 27775 tablet_bootstrap.cc:492] T f4bb31e5c0454e54a7b3855445120783 P 256ce64303ec49e1a5cd8e0ffee436b0: No bootstrap required, opened a new log
I20240502 13:59:54.074784 27782 raft_consensus.cc:504] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.074863 27775 ts_tablet_manager.cc:1367] T f4bb31e5c0454e54a7b3855445120783 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 13:59:54.075227 27784 tablet_bootstrap.cc:492] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68: No bootstrap required, opened a new log
I20240502 13:59:54.075300 27784 ts_tablet_manager.cc:1367] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 13:59:54.075496 27775 raft_consensus.cc:348] T f4bb31e5c0454e54a7b3855445120783 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:54.075632 27775 raft_consensus.cc:374] T f4bb31e5c0454e54a7b3855445120783 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.075667 27775 raft_consensus.cc:729] T f4bb31e5c0454e54a7b3855445120783 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 256ce64303ec49e1a5cd8e0ffee436b0, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.075769 27784 raft_consensus.cc:348] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:54.075815 27775 consensus_queue.cc:260] T f4bb31e5c0454e54a7b3855445120783 P 256ce64303ec49e1a5cd8e0ffee436b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:54.075875 27784 raft_consensus.cc:374] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.075906 27784 raft_consensus.cc:729] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3e36186faa442b7bd87357e74d47f68, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.076022 27784 consensus_queue.cc:260] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:54.076136 27782 leader_election.cc:288] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107), d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841), 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733), 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473)
I20240502 13:59:54.076155 27775 ts_tablet_manager.cc:1398] T f4bb31e5c0454e54a7b3855445120783 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:54.076277 27784 ts_tablet_manager.cc:1398] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:54.076357 27775 tablet_bootstrap.cc:492] T 53a68b75cbce41d69432f21f2cfbf256 P 256ce64303ec49e1a5cd8e0ffee436b0: Bootstrap starting.
I20240502 13:59:54.076501 27784 tablet_bootstrap.cc:492] T abb00d05d9704f5e8a78cb9fd1866e2a P d3e36186faa442b7bd87357e74d47f68: Bootstrap starting.
I20240502 13:59:54.077519 27784 tablet_bootstrap.cc:654] T abb00d05d9704f5e8a78cb9fd1866e2a P d3e36186faa442b7bd87357e74d47f68: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.077554 27775 tablet_bootstrap.cc:654] T 53a68b75cbce41d69432f21f2cfbf256 P 256ce64303ec49e1a5cd8e0ffee436b0: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.078975 27775 tablet_bootstrap.cc:492] T 53a68b75cbce41d69432f21f2cfbf256 P 256ce64303ec49e1a5cd8e0ffee436b0: No bootstrap required, opened a new log
I20240502 13:59:54.079064 27775 ts_tablet_manager.cc:1367] T 53a68b75cbce41d69432f21f2cfbf256 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 13:59:54.079510 27775 raft_consensus.cc:348] T 53a68b75cbce41d69432f21f2cfbf256 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:54.079617 27775 raft_consensus.cc:374] T 53a68b75cbce41d69432f21f2cfbf256 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.079650 27775 raft_consensus.cc:729] T 53a68b75cbce41d69432f21f2cfbf256 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 256ce64303ec49e1a5cd8e0ffee436b0, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.079758 27775 consensus_queue.cc:260] T 53a68b75cbce41d69432f21f2cfbf256 P 256ce64303ec49e1a5cd8e0ffee436b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:54.080037 27775 ts_tablet_manager.cc:1398] T 53a68b75cbce41d69432f21f2cfbf256 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:54.080194 27775 tablet_bootstrap.cc:492] T abb00d05d9704f5e8a78cb9fd1866e2a P 256ce64303ec49e1a5cd8e0ffee436b0: Bootstrap starting.
I20240502 13:59:54.081130 27775 tablet_bootstrap.cc:654] T abb00d05d9704f5e8a78cb9fd1866e2a P 256ce64303ec49e1a5cd8e0ffee436b0: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.082379 27783 tablet_bootstrap.cc:492] T 53a68b75cbce41d69432f21f2cfbf256 P 0280081e4ff74896832ec3985c372b3a: No bootstrap required, opened a new log
I20240502 13:59:54.082470 27783 ts_tablet_manager.cc:1367] T 53a68b75cbce41d69432f21f2cfbf256 P 0280081e4ff74896832ec3985c372b3a: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20240502 13:59:54.083427 27783 raft_consensus.cc:348] T 53a68b75cbce41d69432f21f2cfbf256 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:54.083549 27783 raft_consensus.cc:374] T 53a68b75cbce41d69432f21f2cfbf256 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.083590 27783 raft_consensus.cc:729] T 53a68b75cbce41d69432f21f2cfbf256 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0280081e4ff74896832ec3985c372b3a, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.083732 27783 consensus_queue.cc:260] T 53a68b75cbce41d69432f21f2cfbf256 P 0280081e4ff74896832ec3985c372b3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:54.084122 27783 ts_tablet_manager.cc:1398] T 53a68b75cbce41d69432f21f2cfbf256 P 0280081e4ff74896832ec3985c372b3a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:54.086109 27775 tablet_bootstrap.cc:492] T abb00d05d9704f5e8a78cb9fd1866e2a P 256ce64303ec49e1a5cd8e0ffee436b0: No bootstrap required, opened a new log
I20240502 13:59:54.086213 27775 ts_tablet_manager.cc:1367] T abb00d05d9704f5e8a78cb9fd1866e2a P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 13:59:54.086499 27784 tablet_bootstrap.cc:492] T abb00d05d9704f5e8a78cb9fd1866e2a P d3e36186faa442b7bd87357e74d47f68: No bootstrap required, opened a new log
I20240502 13:59:54.086583 27784 ts_tablet_manager.cc:1367] T abb00d05d9704f5e8a78cb9fd1866e2a P d3e36186faa442b7bd87357e74d47f68: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240502 13:59:54.086659 27775 raft_consensus.cc:348] T abb00d05d9704f5e8a78cb9fd1866e2a P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.086782 27775 raft_consensus.cc:374] T abb00d05d9704f5e8a78cb9fd1866e2a P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.086813 27775 raft_consensus.cc:729] T abb00d05d9704f5e8a78cb9fd1866e2a P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 256ce64303ec49e1a5cd8e0ffee436b0, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.086918 27775 consensus_queue.cc:260] T abb00d05d9704f5e8a78cb9fd1866e2a P 256ce64303ec49e1a5cd8e0ffee436b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.087061 27784 raft_consensus.cc:348] T abb00d05d9704f5e8a78cb9fd1866e2a P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.087174 27775 ts_tablet_manager.cc:1398] T abb00d05d9704f5e8a78cb9fd1866e2a P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:54.087186 27784 raft_consensus.cc:374] T abb00d05d9704f5e8a78cb9fd1866e2a P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.087222 27784 raft_consensus.cc:729] T abb00d05d9704f5e8a78cb9fd1866e2a P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3e36186faa442b7bd87357e74d47f68, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.087337 27784 consensus_queue.cc:260] T abb00d05d9704f5e8a78cb9fd1866e2a P d3e36186faa442b7bd87357e74d47f68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.087661 27784 ts_tablet_manager.cc:1398] T abb00d05d9704f5e8a78cb9fd1866e2a P d3e36186faa442b7bd87357e74d47f68: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:54.088968 26859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abb00d05d9704f5e8a78cb9fd1866e2a" candidate_uuid: "34b161bd719d4d72baf8da4f40aef136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e36186faa442b7bd87357e74d47f68" is_pre_election: true
I20240502 13:59:54.089294 26859 raft_consensus.cc:2454] T abb00d05d9704f5e8a78cb9fd1866e2a P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34b161bd719d4d72baf8da4f40aef136 in term 0.
I20240502 13:59:54.090065 26991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abb00d05d9704f5e8a78cb9fd1866e2a" candidate_uuid: "34b161bd719d4d72baf8da4f40aef136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" is_pre_election: true
I20240502 13:59:54.090185 26991 raft_consensus.cc:2454] T abb00d05d9704f5e8a78cb9fd1866e2a P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34b161bd719d4d72baf8da4f40aef136 in term 0.
I20240502 13:59:54.090502 27387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abb00d05d9704f5e8a78cb9fd1866e2a" candidate_uuid: "34b161bd719d4d72baf8da4f40aef136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44f8c16a14a48a38a5a74af2899c217" is_pre_election: true
I20240502 13:59:54.090698 27387 raft_consensus.cc:2454] T abb00d05d9704f5e8a78cb9fd1866e2a P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34b161bd719d4d72baf8da4f40aef136 in term 0.
I20240502 13:59:54.091001 27452 leader_election.cc:302] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 256ce64303ec49e1a5cd8e0ffee436b0, 34b161bd719d4d72baf8da4f40aef136, d3e36186faa442b7bd87357e74d47f68; no voters:
I20240502 13:59:54.091236 27782 raft_consensus.cc:2781] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:54.091290 27782 raft_consensus.cc:482] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:54.091338 27782 raft_consensus.cc:3037] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.091797 27123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abb00d05d9704f5e8a78cb9fd1866e2a" candidate_uuid: "34b161bd719d4d72baf8da4f40aef136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0280081e4ff74896832ec3985c372b3a" is_pre_election: true
I20240502 13:59:54.092089 27123 raft_consensus.cc:2454] T abb00d05d9704f5e8a78cb9fd1866e2a P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34b161bd719d4d72baf8da4f40aef136 in term 0.
I20240502 13:59:54.104600 27782 raft_consensus.cc:504] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.105413 27782 leader_election.cc:288] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [CANDIDATE]: Term 1 election: Requested vote from peers e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107), d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841), 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733), 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473)
I20240502 13:59:54.105720 27387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abb00d05d9704f5e8a78cb9fd1866e2a" candidate_uuid: "34b161bd719d4d72baf8da4f40aef136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44f8c16a14a48a38a5a74af2899c217"
I20240502 13:59:54.105860 27387 raft_consensus.cc:3037] T abb00d05d9704f5e8a78cb9fd1866e2a P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.105932 27123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abb00d05d9704f5e8a78cb9fd1866e2a" candidate_uuid: "34b161bd719d4d72baf8da4f40aef136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0280081e4ff74896832ec3985c372b3a"
I20240502 13:59:54.106074 27123 raft_consensus.cc:3037] T abb00d05d9704f5e8a78cb9fd1866e2a P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.106796 27387 raft_consensus.cc:2454] T abb00d05d9704f5e8a78cb9fd1866e2a P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34b161bd719d4d72baf8da4f40aef136 in term 1.
I20240502 13:59:54.107097 26859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abb00d05d9704f5e8a78cb9fd1866e2a" candidate_uuid: "34b161bd719d4d72baf8da4f40aef136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e36186faa442b7bd87357e74d47f68"
I20240502 13:59:54.107251 26859 raft_consensus.cc:3037] T abb00d05d9704f5e8a78cb9fd1866e2a P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.107699 26991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abb00d05d9704f5e8a78cb9fd1866e2a" candidate_uuid: "34b161bd719d4d72baf8da4f40aef136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0"
I20240502 13:59:54.107832 26991 raft_consensus.cc:3037] T abb00d05d9704f5e8a78cb9fd1866e2a P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.166800 26991 raft_consensus.cc:2454] T abb00d05d9704f5e8a78cb9fd1866e2a P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34b161bd719d4d72baf8da4f40aef136 in term 1.
I20240502 13:59:54.166800 26859 raft_consensus.cc:2454] T abb00d05d9704f5e8a78cb9fd1866e2a P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34b161bd719d4d72baf8da4f40aef136 in term 1.
I20240502 13:59:54.166801 27123 raft_consensus.cc:2454] T abb00d05d9704f5e8a78cb9fd1866e2a P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34b161bd719d4d72baf8da4f40aef136 in term 1.
I20240502 13:59:54.167356 27452 leader_election.cc:302] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 256ce64303ec49e1a5cd8e0ffee436b0, 34b161bd719d4d72baf8da4f40aef136, e44f8c16a14a48a38a5a74af2899c217; no voters:
I20240502 13:59:54.167578 27782 raft_consensus.cc:2781] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:54.167861 27782 raft_consensus.cc:686] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [term 1 LEADER]: Becoming Leader. State: Replica: 34b161bd719d4d72baf8da4f40aef136, State: Running, Role: LEADER
I20240502 13:59:54.168253 27782 consensus_queue.cc:237] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.171617 26716 catalog_manager.cc:5484] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70). New cstate: current_term: 1 leader_uuid: "34b161bd719d4d72baf8da4f40aef136" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: UNKNOWN } } }
I20240502 13:59:54.175344 27790 raft_consensus.cc:482] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:54.175462 27790 raft_consensus.cc:504] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:54.176766 27790 leader_election.cc:288] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473), 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733), e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107), d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841)
W20240502 13:59:54.179272 26907 tablet.cc:2369] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 13:59:54.182060 27387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4bb31e5c0454e54a7b3855445120783" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44f8c16a14a48a38a5a74af2899c217" is_pre_election: true
I20240502 13:59:54.182054 27123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4bb31e5c0454e54a7b3855445120783" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0280081e4ff74896832ec3985c372b3a" is_pre_election: true
I20240502 13:59:54.182222 27123 raft_consensus.cc:2454] T f4bb31e5c0454e54a7b3855445120783 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 0.
I20240502 13:59:54.182225 27387 raft_consensus.cc:2454] T f4bb31e5c0454e54a7b3855445120783 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 0.
I20240502 13:59:54.182613 26991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4bb31e5c0454e54a7b3855445120783" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" is_pre_election: true
I20240502 13:59:54.182757 26991 raft_consensus.cc:2454] T f4bb31e5c0454e54a7b3855445120783 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 0.
I20240502 13:59:54.182813 27191 leader_election.cc:302] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0280081e4ff74896832ec3985c372b3a, b4fc3c795b1d4505b3a2903532d38b52, e44f8c16a14a48a38a5a74af2899c217; no voters:
I20240502 13:59:54.183074 27790 raft_consensus.cc:2781] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:54.183135 27790 raft_consensus.cc:482] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:54.183173 27790 raft_consensus.cc:3037] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.183207 26859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4bb31e5c0454e54a7b3855445120783" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e36186faa442b7bd87357e74d47f68" is_pre_election: true
I20240502 13:59:54.183315 26859 raft_consensus.cc:2454] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 0.
I20240502 13:59:54.208595 27790 raft_consensus.cc:504] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:54.209203 27790 leader_election.cc:288] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [CANDIDATE]: Term 1 election: Requested vote from peers 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473), 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733), e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107), d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841)
I20240502 13:59:54.209694 26991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4bb31e5c0454e54a7b3855445120783" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0"
I20240502 13:59:54.209851 26991 raft_consensus.cc:3037] T f4bb31e5c0454e54a7b3855445120783 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.209816 26859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4bb31e5c0454e54a7b3855445120783" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e36186faa442b7bd87357e74d47f68"
I20240502 13:59:54.209939 26859 raft_consensus.cc:3037] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.209986 27123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4bb31e5c0454e54a7b3855445120783" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0280081e4ff74896832ec3985c372b3a"
I20240502 13:59:54.210140 27123 raft_consensus.cc:3037] T f4bb31e5c0454e54a7b3855445120783 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.210860 26859 raft_consensus.cc:2454] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 1.
I20240502 13:59:54.210860 26991 raft_consensus.cc:2454] T f4bb31e5c0454e54a7b3855445120783 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 1.
I20240502 13:59:54.211305 27123 raft_consensus.cc:2454] T f4bb31e5c0454e54a7b3855445120783 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 1.
I20240502 13:59:54.211297 27387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4bb31e5c0454e54a7b3855445120783" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44f8c16a14a48a38a5a74af2899c217"
I20240502 13:59:54.211429 27387 raft_consensus.cc:3037] T f4bb31e5c0454e54a7b3855445120783 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.211539 27188 leader_election.cc:302] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 256ce64303ec49e1a5cd8e0ffee436b0, b4fc3c795b1d4505b3a2903532d38b52, d3e36186faa442b7bd87357e74d47f68; no voters:
I20240502 13:59:54.211762 27790 raft_consensus.cc:2781] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:54.212009 27790 raft_consensus.cc:686] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 LEADER]: Becoming Leader. State: Replica: b4fc3c795b1d4505b3a2903532d38b52, State: Running, Role: LEADER
I20240502 13:59:54.212463 27387 raft_consensus.cc:2454] T f4bb31e5c0454e54a7b3855445120783 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 1.
I20240502 13:59:54.212419 27790 consensus_queue.cc:237] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:54.215173 26716 catalog_manager.cc:5484] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68). New cstate: current_term: 1 leader_uuid: "b4fc3c795b1d4505b3a2903532d38b52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: UNKNOWN } } }
I20240502 13:59:54.245851 27794 raft_consensus.cc:482] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:54.246007 27794 raft_consensus.cc:504] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:54.247254 27794 leader_election.cc:288] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733), 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473), b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68:35167), 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217)
I20240502 13:59:54.252266 26991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53a68b75cbce41d69432f21f2cfbf256" candidate_uuid: "d3e36186faa442b7bd87357e74d47f68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" is_pre_election: true
I20240502 13:59:54.252470 26991 raft_consensus.cc:2454] T 53a68b75cbce41d69432f21f2cfbf256 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3e36186faa442b7bd87357e74d47f68 in term 0.
I20240502 13:59:54.252599 27519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53a68b75cbce41d69432f21f2cfbf256" candidate_uuid: "d3e36186faa442b7bd87357e74d47f68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b161bd719d4d72baf8da4f40aef136" is_pre_election: true
I20240502 13:59:54.252730 27519 raft_consensus.cc:2454] T 53a68b75cbce41d69432f21f2cfbf256 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3e36186faa442b7bd87357e74d47f68 in term 0.
I20240502 13:59:54.253131 27255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53a68b75cbce41d69432f21f2cfbf256" candidate_uuid: "d3e36186faa442b7bd87357e74d47f68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4fc3c795b1d4505b3a2903532d38b52" is_pre_election: true
I20240502 13:59:54.253140 27123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53a68b75cbce41d69432f21f2cfbf256" candidate_uuid: "d3e36186faa442b7bd87357e74d47f68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0280081e4ff74896832ec3985c372b3a" is_pre_election: true
I20240502 13:59:54.253279 27255 raft_consensus.cc:2454] T 53a68b75cbce41d69432f21f2cfbf256 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3e36186faa442b7bd87357e74d47f68 in term 0.
I20240502 13:59:54.253239 26792 leader_election.cc:302] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 256ce64303ec49e1a5cd8e0ffee436b0, 34b161bd719d4d72baf8da4f40aef136, d3e36186faa442b7bd87357e74d47f68; no voters:
I20240502 13:59:54.253296 27123 raft_consensus.cc:2454] T 53a68b75cbce41d69432f21f2cfbf256 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3e36186faa442b7bd87357e74d47f68 in term 0.
I20240502 13:59:54.253515 27794 raft_consensus.cc:2781] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:54.253592 27794 raft_consensus.cc:482] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:54.253635 27794 raft_consensus.cc:3037] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Advancing to term 1
W20240502 13:59:54.276118 27171 tablet.cc:2369] T 53a68b75cbce41d69432f21f2cfbf256 P 0280081e4ff74896832ec3985c372b3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 13:59:54.307186 27794 raft_consensus.cc:504] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:54.307807 27794 leader_election.cc:288] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [CANDIDATE]: Term 1 election: Requested vote from peers 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733), 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473), b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68:35167), 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217)
I20240502 13:59:54.308429 27123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53a68b75cbce41d69432f21f2cfbf256" candidate_uuid: "d3e36186faa442b7bd87357e74d47f68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0280081e4ff74896832ec3985c372b3a"
I20240502 13:59:54.308442 26991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53a68b75cbce41d69432f21f2cfbf256" candidate_uuid: "d3e36186faa442b7bd87357e74d47f68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0"
I20240502 13:59:54.308501 27519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53a68b75cbce41d69432f21f2cfbf256" candidate_uuid: "d3e36186faa442b7bd87357e74d47f68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b161bd719d4d72baf8da4f40aef136"
I20240502 13:59:54.308575 27123 raft_consensus.cc:3037] T 53a68b75cbce41d69432f21f2cfbf256 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.308624 27519 raft_consensus.cc:3037] T 53a68b75cbce41d69432f21f2cfbf256 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.308624 26991 raft_consensus.cc:3037] T 53a68b75cbce41d69432f21f2cfbf256 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.309559 27123 raft_consensus.cc:2454] T 53a68b75cbce41d69432f21f2cfbf256 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3e36186faa442b7bd87357e74d47f68 in term 1.
I20240502 13:59:54.309559 27519 raft_consensus.cc:2454] T 53a68b75cbce41d69432f21f2cfbf256 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3e36186faa442b7bd87357e74d47f68 in term 1.
I20240502 13:59:54.309854 27255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53a68b75cbce41d69432f21f2cfbf256" candidate_uuid: "d3e36186faa442b7bd87357e74d47f68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4fc3c795b1d4505b3a2903532d38b52"
I20240502 13:59:54.309979 26991 raft_consensus.cc:2454] T 53a68b75cbce41d69432f21f2cfbf256 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3e36186faa442b7bd87357e74d47f68 in term 1.
I20240502 13:59:54.310024 27255 raft_consensus.cc:3037] T 53a68b75cbce41d69432f21f2cfbf256 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.310408 26795 leader_election.cc:302] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0280081e4ff74896832ec3985c372b3a, 34b161bd719d4d72baf8da4f40aef136, d3e36186faa442b7bd87357e74d47f68; no voters:
I20240502 13:59:54.310627 27794 raft_consensus.cc:2781] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:54.310943 27794 raft_consensus.cc:686] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [term 1 LEADER]: Becoming Leader. State: Replica: d3e36186faa442b7bd87357e74d47f68, State: Running, Role: LEADER
I20240502 13:59:54.311312 27794 consensus_queue.cc:237] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:54.314675 26716 catalog_manager.cc:5484] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3e36186faa442b7bd87357e74d47f68 (127.24.66.65). New cstate: current_term: 1 leader_uuid: "d3e36186faa442b7bd87357e74d47f68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: UNKNOWN } } }
I20240502 13:59:54.406309 27255 raft_consensus.cc:2454] T 53a68b75cbce41d69432f21f2cfbf256 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3e36186faa442b7bd87357e74d47f68 in term 1.
W20240502 13:59:54.455082 27824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:54.455402 27824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:54.459352 27824 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240502 13:59:54.540271 27519 raft_consensus.cc:1261] T 53a68b75cbce41d69432f21f2cfbf256 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer d3e36186faa442b7bd87357e74d47f68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:54.540763 26991 raft_consensus.cc:1261] T 53a68b75cbce41d69432f21f2cfbf256 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer d3e36186faa442b7bd87357e74d47f68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:54.541476 27123 raft_consensus.cc:1261] T 53a68b75cbce41d69432f21f2cfbf256 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer d3e36186faa442b7bd87357e74d47f68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:54.541986 27255 raft_consensus.cc:1261] T 53a68b75cbce41d69432f21f2cfbf256 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer d3e36186faa442b7bd87357e74d47f68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:54.543273 26859 raft_consensus.cc:1261] T abb00d05d9704f5e8a78cb9fd1866e2a P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:54.543884 27387 raft_consensus.cc:1261] T abb00d05d9704f5e8a78cb9fd1866e2a P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:54.543905 27808 consensus_queue.cc:1035] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:54.544505 27808 consensus_queue.cc:1035] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:54.544821 27822 consensus_queue.cc:1035] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:54.545239 27822 consensus_queue.cc:1035] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:54.545533 27822 consensus_queue.cc:1035] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:54.545610 26991 raft_consensus.cc:1261] T abb00d05d9704f5e8a78cb9fd1866e2a P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:54.545827 27822 consensus_queue.cc:1035] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:54.546367 27808 consensus_queue.cc:1035] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:54.553043 27123 raft_consensus.cc:1261] T abb00d05d9704f5e8a78cb9fd1866e2a P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:54.553436 27808 consensus_queue.cc:1035] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:54.559688 27386 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:54.560070 26858 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:54.560536 27814 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:54.561129 27123 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:54.561609 27814 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:54.561651 26989 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:54.561713 27790 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:54.563150 27814 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:54.577741 27851 mvcc.cc:205] Tried to move back new op lower bound from 7023240784040546304 to 7023240782517493760. Current Snapshot: MvccSnapshot[applied={T|T < 7023240784040546304}]
I20240502 13:59:54.582391 27858 mvcc.cc:205] Tried to move back new op lower bound from 7023240784040546304 to 7023240782517493760. Current Snapshot: MvccSnapshot[applied={T|T < 7023240784040546304}]
I20240502 13:59:54.585052 27852 mvcc.cc:205] Tried to move back new op lower bound from 7023240784060411904 to 7023240782517493760. Current Snapshot: MvccSnapshot[applied={T|T < 7023240784040546304}]
I20240502 13:59:54.585608 27853 mvcc.cc:205] Tried to move back new op lower bound from 7023240784048648192 to 7023240783102255104. Current Snapshot: MvccSnapshot[applied={T|T < 7023240784023089152}]
I20240502 13:59:54.588567 27874 mvcc.cc:205] Tried to move back new op lower bound from 7023240784048648192 to 7023240783102255104. Current Snapshot: MvccSnapshot[applied={T|T < 7023240784023089152}]
I20240502 13:59:54.638476 27869 mvcc.cc:205] Tried to move back new op lower bound from 7023240784150450176 to 7023240782697406464. Current Snapshot: MvccSnapshot[applied={T|T < 7023240784150450176}]
INSERT report
rows total: 2000
time total: 155.422 ms
time per row: 0.077711 ms
I20240502 13:59:54.661132 26714 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50400:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 13:59:54.666342 26837 tablet_service.cc:1467] Processing CreateTablet for tablet 4a74c0c312a7403d89958a3632cab7d3 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a35cd53b479c486fad6a93e2f4317542]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.666420 27102 tablet_service.cc:1467] Processing CreateTablet for tablet 4a74c0c312a7403d89958a3632cab7d3 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a35cd53b479c486fad6a93e2f4317542]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.666522 27499 tablet_service.cc:1467] Processing CreateTablet for tablet 4a74c0c312a7403d89958a3632cab7d3 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a35cd53b479c486fad6a93e2f4317542]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.666672 26837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a74c0c312a7403d89958a3632cab7d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.666702 27102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a74c0c312a7403d89958a3632cab7d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.666759 27499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a74c0c312a7403d89958a3632cab7d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.668040 27498 tablet_service.cc:1467] Processing CreateTablet for tablet 95f064e56ee241dba95f9a6cd5984d97 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a35cd53b479c486fad6a93e2f4317542]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.668247 27498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95f064e56ee241dba95f9a6cd5984d97. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.668629 26969 tablet_service.cc:1467] Processing CreateTablet for tablet 41e73be38c4348a5bb86da6c0f6c6d22 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a35cd53b479c486fad6a93e2f4317542]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.668910 26969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41e73be38c4348a5bb86da6c0f6c6d22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.669245 27367 tablet_service.cc:1467] Processing CreateTablet for tablet 4a74c0c312a7403d89958a3632cab7d3 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a35cd53b479c486fad6a93e2f4317542]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.669476 27367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a74c0c312a7403d89958a3632cab7d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.669798 27235 tablet_service.cc:1467] Processing CreateTablet for tablet 95f064e56ee241dba95f9a6cd5984d97 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a35cd53b479c486fad6a93e2f4317542]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.670058 27235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95f064e56ee241dba95f9a6cd5984d97. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.670708 27906 tablet_bootstrap.cc:492] T 4a74c0c312a7403d89958a3632cab7d3 P d3e36186faa442b7bd87357e74d47f68: Bootstrap starting.
I20240502 13:59:54.671207 27234 tablet_service.cc:1467] Processing CreateTablet for tablet 4a74c0c312a7403d89958a3632cab7d3 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a35cd53b479c486fad6a93e2f4317542]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.671424 27234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a74c0c312a7403d89958a3632cab7d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.671747 27233 tablet_service.cc:1467] Processing CreateTablet for tablet 41e73be38c4348a5bb86da6c0f6c6d22 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a35cd53b479c486fad6a93e2f4317542]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.671888 27906 tablet_bootstrap.cc:654] T 4a74c0c312a7403d89958a3632cab7d3 P d3e36186faa442b7bd87357e74d47f68: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.671960 27233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41e73be38c4348a5bb86da6c0f6c6d22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.672612 27103 tablet_service.cc:1467] Processing CreateTablet for tablet 95f064e56ee241dba95f9a6cd5984d97 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a35cd53b479c486fad6a93e2f4317542]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.672852 27103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95f064e56ee241dba95f9a6cd5984d97. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.672959 27906 tablet_bootstrap.cc:492] T 4a74c0c312a7403d89958a3632cab7d3 P d3e36186faa442b7bd87357e74d47f68: No bootstrap required, opened a new log
I20240502 13:59:54.673029 27906 ts_tablet_manager.cc:1367] T 4a74c0c312a7403d89958a3632cab7d3 P d3e36186faa442b7bd87357e74d47f68: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:54.673106 27366 tablet_service.cc:1467] Processing CreateTablet for tablet 41e73be38c4348a5bb86da6c0f6c6d22 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a35cd53b479c486fad6a93e2f4317542]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.673324 27366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41e73be38c4348a5bb86da6c0f6c6d22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.673427 27906 raft_consensus.cc:348] T 4a74c0c312a7403d89958a3632cab7d3 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.673545 27906 raft_consensus.cc:374] T 4a74c0c312a7403d89958a3632cab7d3 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.673576 27906 raft_consensus.cc:729] T 4a74c0c312a7403d89958a3632cab7d3 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3e36186faa442b7bd87357e74d47f68, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.673708 27906 consensus_queue.cc:260] T 4a74c0c312a7403d89958a3632cab7d3 P d3e36186faa442b7bd87357e74d47f68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.674036 27906 ts_tablet_manager.cc:1398] T 4a74c0c312a7403d89958a3632cab7d3 P d3e36186faa442b7bd87357e74d47f68: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:54.674825 27101 tablet_service.cc:1467] Processing CreateTablet for tablet 41e73be38c4348a5bb86da6c0f6c6d22 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a35cd53b479c486fad6a93e2f4317542]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.675047 27101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41e73be38c4348a5bb86da6c0f6c6d22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.675611 26837 tablet_service.cc:1467] Processing CreateTablet for tablet 95f064e56ee241dba95f9a6cd5984d97 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a35cd53b479c486fad6a93e2f4317542]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.675828 26837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95f064e56ee241dba95f9a6cd5984d97. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.676604 26838 tablet_service.cc:1467] Processing CreateTablet for tablet 41e73be38c4348a5bb86da6c0f6c6d22 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a35cd53b479c486fad6a93e2f4317542]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.676849 26838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41e73be38c4348a5bb86da6c0f6c6d22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.677206 27909 tablet_bootstrap.cc:492] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52: Bootstrap starting.
I20240502 13:59:54.677835 27906 tablet_bootstrap.cc:492] T 95f064e56ee241dba95f9a6cd5984d97 P d3e36186faa442b7bd87357e74d47f68: Bootstrap starting.
I20240502 13:59:54.678130 27909 tablet_bootstrap.cc:654] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.678730 27906 tablet_bootstrap.cc:654] T 95f064e56ee241dba95f9a6cd5984d97 P d3e36186faa442b7bd87357e74d47f68: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.678786 27911 tablet_bootstrap.cc:492] T 95f064e56ee241dba95f9a6cd5984d97 P 0280081e4ff74896832ec3985c372b3a: Bootstrap starting.
I20240502 13:59:54.679632 27911 tablet_bootstrap.cc:654] T 95f064e56ee241dba95f9a6cd5984d97 P 0280081e4ff74896832ec3985c372b3a: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.679865 27906 tablet_bootstrap.cc:492] T 95f064e56ee241dba95f9a6cd5984d97 P d3e36186faa442b7bd87357e74d47f68: No bootstrap required, opened a new log
I20240502 13:59:54.679841 27367 tablet_service.cc:1467] Processing CreateTablet for tablet 95f064e56ee241dba95f9a6cd5984d97 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a35cd53b479c486fad6a93e2f4317542]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:54.679941 27906 ts_tablet_manager.cc:1367] T 95f064e56ee241dba95f9a6cd5984d97 P d3e36186faa442b7bd87357e74d47f68: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:54.680058 27367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95f064e56ee241dba95f9a6cd5984d97. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:54.680533 27906 raft_consensus.cc:348] T 95f064e56ee241dba95f9a6cd5984d97 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.680636 27906 raft_consensus.cc:374] T 95f064e56ee241dba95f9a6cd5984d97 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.680672 27906 raft_consensus.cc:729] T 95f064e56ee241dba95f9a6cd5984d97 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3e36186faa442b7bd87357e74d47f68, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.680800 27906 consensus_queue.cc:260] T 95f064e56ee241dba95f9a6cd5984d97 P d3e36186faa442b7bd87357e74d47f68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.680943 27911 tablet_bootstrap.cc:492] T 95f064e56ee241dba95f9a6cd5984d97 P 0280081e4ff74896832ec3985c372b3a: No bootstrap required, opened a new log
I20240502 13:59:54.681016 27911 ts_tablet_manager.cc:1367] T 95f064e56ee241dba95f9a6cd5984d97 P 0280081e4ff74896832ec3985c372b3a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:54.681089 27906 ts_tablet_manager.cc:1398] T 95f064e56ee241dba95f9a6cd5984d97 P d3e36186faa442b7bd87357e74d47f68: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:54.681242 27906 tablet_bootstrap.cc:492] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68: Bootstrap starting.
I20240502 13:59:54.681419 27911 raft_consensus.cc:348] T 95f064e56ee241dba95f9a6cd5984d97 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.681522 27911 raft_consensus.cc:374] T 95f064e56ee241dba95f9a6cd5984d97 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.681558 27911 raft_consensus.cc:729] T 95f064e56ee241dba95f9a6cd5984d97 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0280081e4ff74896832ec3985c372b3a, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.681707 27911 consensus_queue.cc:260] T 95f064e56ee241dba95f9a6cd5984d97 P 0280081e4ff74896832ec3985c372b3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.682000 27911 ts_tablet_manager.cc:1398] T 95f064e56ee241dba95f9a6cd5984d97 P 0280081e4ff74896832ec3985c372b3a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:54.682085 27906 tablet_bootstrap.cc:654] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.682170 27911 tablet_bootstrap.cc:492] T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a: Bootstrap starting.
I20240502 13:59:54.683050 27911 tablet_bootstrap.cc:654] T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.683161 27906 tablet_bootstrap.cc:492] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68: No bootstrap required, opened a new log
I20240502 13:59:54.683236 27906 ts_tablet_manager.cc:1367] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:54.683684 27906 raft_consensus.cc:348] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.683787 27906 raft_consensus.cc:374] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.683818 27906 raft_consensus.cc:729] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3e36186faa442b7bd87357e74d47f68, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.683920 27906 consensus_queue.cc:260] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.684193 27906 ts_tablet_manager.cc:1398] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:54.684998 27911 tablet_bootstrap.cc:492] T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a: No bootstrap required, opened a new log
I20240502 13:59:54.685086 27911 ts_tablet_manager.cc:1367] T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 13:59:54.685434 27907 tablet_bootstrap.cc:492] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0: Bootstrap starting.
I20240502 13:59:54.685513 27911 raft_consensus.cc:348] T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.685612 27911 raft_consensus.cc:374] T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.685647 27911 raft_consensus.cc:729] T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0280081e4ff74896832ec3985c372b3a, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.685763 27911 consensus_queue.cc:260] T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.686028 27911 ts_tablet_manager.cc:1398] T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:54.686208 27911 tablet_bootstrap.cc:492] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a: Bootstrap starting.
I20240502 13:59:54.687273 27911 tablet_bootstrap.cc:654] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.687865 27908 tablet_bootstrap.cc:492] T 4a74c0c312a7403d89958a3632cab7d3 P e44f8c16a14a48a38a5a74af2899c217: Bootstrap starting.
I20240502 13:59:54.688410 27909 tablet_bootstrap.cc:492] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52: No bootstrap required, opened a new log
I20240502 13:59:54.688519 27909 ts_tablet_manager.cc:1367] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 13:59:54.688747 27908 tablet_bootstrap.cc:654] T 4a74c0c312a7403d89958a3632cab7d3 P e44f8c16a14a48a38a5a74af2899c217: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.688964 27909 raft_consensus.cc:348] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.689082 27909 raft_consensus.cc:374] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.689133 27909 raft_consensus.cc:729] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4fc3c795b1d4505b3a2903532d38b52, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.689182 27907 tablet_bootstrap.cc:654] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.689275 27909 consensus_queue.cc:260] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.689386 27905 tablet_bootstrap.cc:492] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136: Bootstrap starting.
I20240502 13:59:54.689570 27909 ts_tablet_manager.cc:1398] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:54.689780 27909 tablet_bootstrap.cc:492] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52: Bootstrap starting.
I20240502 13:59:54.690598 27905 tablet_bootstrap.cc:654] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.690616 27909 tablet_bootstrap.cc:654] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.691566 27911 tablet_bootstrap.cc:492] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a: No bootstrap required, opened a new log
I20240502 13:59:54.691653 27911 ts_tablet_manager.cc:1367] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 13:59:54.691797 27905 tablet_bootstrap.cc:492] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136: No bootstrap required, opened a new log
I20240502 13:59:54.691890 27905 ts_tablet_manager.cc:1367] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 13:59:54.692134 27911 raft_consensus.cc:348] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.692261 27911 raft_consensus.cc:374] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.692312 27911 raft_consensus.cc:729] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0280081e4ff74896832ec3985c372b3a, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.692356 27905 raft_consensus.cc:348] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.692476 27905 raft_consensus.cc:374] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.692461 27911 consensus_queue.cc:260] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.692524 27905 raft_consensus.cc:729] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34b161bd719d4d72baf8da4f40aef136, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.692684 27905 consensus_queue.cc:260] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.692770 27911 ts_tablet_manager.cc:1398] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:54.693145 27905 ts_tablet_manager.cc:1398] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:54.694319 27905 tablet_bootstrap.cc:492] T 95f064e56ee241dba95f9a6cd5984d97 P 34b161bd719d4d72baf8da4f40aef136: Bootstrap starting.
I20240502 13:59:54.694754 27909 tablet_bootstrap.cc:492] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52: No bootstrap required, opened a new log
I20240502 13:59:54.694842 27909 ts_tablet_manager.cc:1367] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 13:59:54.695279 27909 raft_consensus.cc:348] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.695384 27905 tablet_bootstrap.cc:654] T 95f064e56ee241dba95f9a6cd5984d97 P 34b161bd719d4d72baf8da4f40aef136: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.695413 27909 raft_consensus.cc:374] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.695451 27909 raft_consensus.cc:729] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4fc3c795b1d4505b3a2903532d38b52, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.695597 27909 consensus_queue.cc:260] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.695883 27909 ts_tablet_manager.cc:1398] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:54.696072 27909 tablet_bootstrap.cc:492] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52: Bootstrap starting.
I20240502 13:59:54.696079 27907 tablet_bootstrap.cc:492] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0: No bootstrap required, opened a new log
I20240502 13:59:54.696255 27907 ts_tablet_manager.cc:1367] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20240502 13:59:54.696802 27907 raft_consensus.cc:348] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.696938 27908 tablet_bootstrap.cc:492] T 4a74c0c312a7403d89958a3632cab7d3 P e44f8c16a14a48a38a5a74af2899c217: No bootstrap required, opened a new log
I20240502 13:59:54.697027 27908 ts_tablet_manager.cc:1367] T 4a74c0c312a7403d89958a3632cab7d3 P e44f8c16a14a48a38a5a74af2899c217: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 13:59:54.697016 27907 raft_consensus.cc:374] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.697109 27907 raft_consensus.cc:729] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 256ce64303ec49e1a5cd8e0ffee436b0, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.697181 27909 tablet_bootstrap.cc:654] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.697356 27907 consensus_queue.cc:260] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.697705 27908 raft_consensus.cc:348] T 4a74c0c312a7403d89958a3632cab7d3 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.697818 27908 raft_consensus.cc:374] T 4a74c0c312a7403d89958a3632cab7d3 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.697851 27908 raft_consensus.cc:729] T 4a74c0c312a7403d89958a3632cab7d3 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44f8c16a14a48a38a5a74af2899c217, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.698429 27907 ts_tablet_manager.cc:1398] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:54.698949 27909 tablet_bootstrap.cc:492] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52: No bootstrap required, opened a new log
I20240502 13:59:54.699038 27909 ts_tablet_manager.cc:1367] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 13:59:54.699062 27905 tablet_bootstrap.cc:492] T 95f064e56ee241dba95f9a6cd5984d97 P 34b161bd719d4d72baf8da4f40aef136: No bootstrap required, opened a new log
I20240502 13:59:54.699159 27905 ts_tablet_manager.cc:1367] T 95f064e56ee241dba95f9a6cd5984d97 P 34b161bd719d4d72baf8da4f40aef136: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 13:59:54.699457 27908 consensus_queue.cc:260] T 4a74c0c312a7403d89958a3632cab7d3 P e44f8c16a14a48a38a5a74af2899c217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.699502 27909 raft_consensus.cc:348] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.699587 27909 raft_consensus.cc:374] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.699625 27909 raft_consensus.cc:729] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4fc3c795b1d4505b3a2903532d38b52, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.699613 27905 raft_consensus.cc:348] T 95f064e56ee241dba95f9a6cd5984d97 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.699736 27905 raft_consensus.cc:374] T 95f064e56ee241dba95f9a6cd5984d97 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.699790 27905 raft_consensus.cc:729] T 95f064e56ee241dba95f9a6cd5984d97 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34b161bd719d4d72baf8da4f40aef136, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.699733 27909 consensus_queue.cc:260] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.699908 27905 consensus_queue.cc:260] T 95f064e56ee241dba95f9a6cd5984d97 P 34b161bd719d4d72baf8da4f40aef136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.700026 27909 ts_tablet_manager.cc:1398] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 13:59:54.700186 27905 ts_tablet_manager.cc:1398] T 95f064e56ee241dba95f9a6cd5984d97 P 34b161bd719d4d72baf8da4f40aef136: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:54.700191 27908 ts_tablet_manager.cc:1398] T 4a74c0c312a7403d89958a3632cab7d3 P e44f8c16a14a48a38a5a74af2899c217: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 13:59:54.700450 27908 tablet_bootstrap.cc:492] T 41e73be38c4348a5bb86da6c0f6c6d22 P e44f8c16a14a48a38a5a74af2899c217: Bootstrap starting.
I20240502 13:59:54.701320 27908 tablet_bootstrap.cc:654] T 41e73be38c4348a5bb86da6c0f6c6d22 P e44f8c16a14a48a38a5a74af2899c217: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.702725 27908 tablet_bootstrap.cc:492] T 41e73be38c4348a5bb86da6c0f6c6d22 P e44f8c16a14a48a38a5a74af2899c217: No bootstrap required, opened a new log
I20240502 13:59:54.702824 27908 ts_tablet_manager.cc:1367] T 41e73be38c4348a5bb86da6c0f6c6d22 P e44f8c16a14a48a38a5a74af2899c217: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:54.703266 27908 raft_consensus.cc:348] T 41e73be38c4348a5bb86da6c0f6c6d22 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.703401 27908 raft_consensus.cc:374] T 41e73be38c4348a5bb86da6c0f6c6d22 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.703446 27908 raft_consensus.cc:729] T 41e73be38c4348a5bb86da6c0f6c6d22 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44f8c16a14a48a38a5a74af2899c217, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.703580 27908 consensus_queue.cc:260] T 41e73be38c4348a5bb86da6c0f6c6d22 P e44f8c16a14a48a38a5a74af2899c217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.703840 27908 ts_tablet_manager.cc:1398] T 41e73be38c4348a5bb86da6c0f6c6d22 P e44f8c16a14a48a38a5a74af2899c217: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:54.704010 27908 tablet_bootstrap.cc:492] T 95f064e56ee241dba95f9a6cd5984d97 P e44f8c16a14a48a38a5a74af2899c217: Bootstrap starting.
I20240502 13:59:54.704922 27908 tablet_bootstrap.cc:654] T 95f064e56ee241dba95f9a6cd5984d97 P e44f8c16a14a48a38a5a74af2899c217: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:54.706246 27908 tablet_bootstrap.cc:492] T 95f064e56ee241dba95f9a6cd5984d97 P e44f8c16a14a48a38a5a74af2899c217: No bootstrap required, opened a new log
I20240502 13:59:54.706334 27908 ts_tablet_manager.cc:1367] T 95f064e56ee241dba95f9a6cd5984d97 P e44f8c16a14a48a38a5a74af2899c217: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:54.706771 27908 raft_consensus.cc:348] T 95f064e56ee241dba95f9a6cd5984d97 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.706880 27908 raft_consensus.cc:374] T 95f064e56ee241dba95f9a6cd5984d97 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:54.706916 27908 raft_consensus.cc:729] T 95f064e56ee241dba95f9a6cd5984d97 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44f8c16a14a48a38a5a74af2899c217, State: Initialized, Role: FOLLOWER
I20240502 13:59:54.707018 27908 consensus_queue.cc:260] T 95f064e56ee241dba95f9a6cd5984d97 P e44f8c16a14a48a38a5a74af2899c217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.707265 27908 ts_tablet_manager.cc:1398] T 95f064e56ee241dba95f9a6cd5984d97 P e44f8c16a14a48a38a5a74af2899c217: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:54.735664 27797 raft_consensus.cc:482] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:54.735808 27797 raft_consensus.cc:504] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.737093 27797 leader_election.cc:288] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841), 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733), b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68:35167), e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107)
I20240502 13:59:54.742385 27123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e73be38c4348a5bb86da6c0f6c6d22" candidate_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0280081e4ff74896832ec3985c372b3a" is_pre_election: true
I20240502 13:59:54.742563 27123 raft_consensus.cc:2454] T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 256ce64303ec49e1a5cd8e0ffee436b0 in term 0.
I20240502 13:59:54.742602 26859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e73be38c4348a5bb86da6c0f6c6d22" candidate_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e36186faa442b7bd87357e74d47f68" is_pre_election: true
I20240502 13:59:54.742780 26859 raft_consensus.cc:2454] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 256ce64303ec49e1a5cd8e0ffee436b0 in term 0.
I20240502 13:59:54.743013 27255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e73be38c4348a5bb86da6c0f6c6d22" candidate_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4fc3c795b1d4505b3a2903532d38b52" is_pre_election: true
I20240502 13:59:54.743172 27255 raft_consensus.cc:2454] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 256ce64303ec49e1a5cd8e0ffee436b0 in term 0.
I20240502 13:59:54.743268 26924 leader_election.cc:302] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0280081e4ff74896832ec3985c372b3a, 256ce64303ec49e1a5cd8e0ffee436b0, d3e36186faa442b7bd87357e74d47f68; no voters:
I20240502 13:59:54.743530 27797 raft_consensus.cc:2781] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:54.743515 27386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e73be38c4348a5bb86da6c0f6c6d22" candidate_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44f8c16a14a48a38a5a74af2899c217" is_pre_election: true
I20240502 13:59:54.743590 27797 raft_consensus.cc:482] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:54.743638 27797 raft_consensus.cc:3037] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.743644 27386 raft_consensus.cc:2454] T 41e73be38c4348a5bb86da6c0f6c6d22 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 256ce64303ec49e1a5cd8e0ffee436b0 in term 0.
I20240502 13:59:54.762177 27791 raft_consensus.cc:482] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:54.762305 27791 raft_consensus.cc:504] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.763510 27791 leader_election.cc:288] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841), 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217), e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107), b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68:35167)
I20240502 13:59:54.768388 27255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a74c0c312a7403d89958a3632cab7d3" candidate_uuid: "0280081e4ff74896832ec3985c372b3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4fc3c795b1d4505b3a2903532d38b52" is_pre_election: true
I20240502 13:59:54.768379 26859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a74c0c312a7403d89958a3632cab7d3" candidate_uuid: "0280081e4ff74896832ec3985c372b3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e36186faa442b7bd87357e74d47f68" is_pre_election: true
I20240502 13:59:54.768448 27386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a74c0c312a7403d89958a3632cab7d3" candidate_uuid: "0280081e4ff74896832ec3985c372b3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44f8c16a14a48a38a5a74af2899c217" is_pre_election: true
I20240502 13:59:54.768560 27386 raft_consensus.cc:2454] T 4a74c0c312a7403d89958a3632cab7d3 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0280081e4ff74896832ec3985c372b3a in term 0.
I20240502 13:59:54.768556 27255 raft_consensus.cc:2454] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0280081e4ff74896832ec3985c372b3a in term 0.
I20240502 13:59:54.768566 26859 raft_consensus.cc:2454] T 4a74c0c312a7403d89958a3632cab7d3 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0280081e4ff74896832ec3985c372b3a in term 0.
I20240502 13:59:54.769006 27056 leader_election.cc:302] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 5 voters: 4 yes votes; 0 no votes. yes voters: 0280081e4ff74896832ec3985c372b3a, b4fc3c795b1d4505b3a2903532d38b52, d3e36186faa442b7bd87357e74d47f68, e44f8c16a14a48a38a5a74af2899c217; no voters:
I20240502 13:59:54.769213 27791 raft_consensus.cc:2781] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:54.769272 27791 raft_consensus.cc:482] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:54.769316 27791 raft_consensus.cc:3037] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.769547 27519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a74c0c312a7403d89958a3632cab7d3" candidate_uuid: "0280081e4ff74896832ec3985c372b3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b161bd719d4d72baf8da4f40aef136" is_pre_election: true
I20240502 13:59:54.769685 27519 raft_consensus.cc:2454] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0280081e4ff74896832ec3985c372b3a in term 0.
I20240502 13:59:54.802284 27896 raft_consensus.cc:482] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:54.802431 27896 raft_consensus.cc:504] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.803141 27896 leader_election.cc:288] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107), 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217), d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841), 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733)
I20240502 13:59:54.803434 27386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95f064e56ee241dba95f9a6cd5984d97" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44f8c16a14a48a38a5a74af2899c217" is_pre_election: true
I20240502 13:59:54.803584 27386 raft_consensus.cc:2454] T 95f064e56ee241dba95f9a6cd5984d97 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 0.
I20240502 13:59:54.803812 27123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95f064e56ee241dba95f9a6cd5984d97" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0280081e4ff74896832ec3985c372b3a" is_pre_election: true
I20240502 13:59:54.803959 27123 raft_consensus.cc:2454] T 95f064e56ee241dba95f9a6cd5984d97 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 0.
I20240502 13:59:54.804124 26859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95f064e56ee241dba95f9a6cd5984d97" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e36186faa442b7bd87357e74d47f68" is_pre_election: true
I20240502 13:59:54.804262 26859 raft_consensus.cc:2454] T 95f064e56ee241dba95f9a6cd5984d97 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 0.
I20240502 13:59:54.804270 27191 leader_election.cc:302] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0280081e4ff74896832ec3985c372b3a, b4fc3c795b1d4505b3a2903532d38b52, e44f8c16a14a48a38a5a74af2899c217; no voters:
I20240502 13:59:54.804466 27896 raft_consensus.cc:2781] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:54.804527 27896 raft_consensus.cc:482] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:54.804566 27896 raft_consensus.cc:3037] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.808099 27519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95f064e56ee241dba95f9a6cd5984d97" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b161bd719d4d72baf8da4f40aef136" is_pre_election: true
I20240502 13:59:54.808228 27519 raft_consensus.cc:2454] T 95f064e56ee241dba95f9a6cd5984d97 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 0.
I20240502 13:59:54.812960 27797 raft_consensus.cc:504] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.812960 27791 raft_consensus.cc:504] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.812955 27896 raft_consensus.cc:504] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.813558 27791 leader_election.cc:288] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [CANDIDATE]: Term 1 election: Requested vote from peers d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841), 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217), e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107), b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68:35167)
I20240502 13:59:54.813568 27797 leader_election.cc:288] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [CANDIDATE]: Term 1 election: Requested vote from peers d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841), 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733), b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68:35167), e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107)
I20240502 13:59:54.813635 27896 leader_election.cc:288] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [CANDIDATE]: Term 1 election: Requested vote from peers e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107), 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217), d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841), 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733)
I20240502 13:59:54.814296 27519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a74c0c312a7403d89958a3632cab7d3" candidate_uuid: "0280081e4ff74896832ec3985c372b3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b161bd719d4d72baf8da4f40aef136"
I20240502 13:59:54.814337 26859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e73be38c4348a5bb86da6c0f6c6d22" candidate_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e36186faa442b7bd87357e74d47f68"
I20240502 13:59:54.814435 27519 raft_consensus.cc:3037] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.814476 26859 raft_consensus.cc:3037] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.814664 27386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e73be38c4348a5bb86da6c0f6c6d22" candidate_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44f8c16a14a48a38a5a74af2899c217"
I20240502 13:59:54.814785 27386 raft_consensus.cc:3037] T 41e73be38c4348a5bb86da6c0f6c6d22 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.815521 27519 raft_consensus.cc:2454] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0280081e4ff74896832ec3985c372b3a in term 1.
I20240502 13:59:54.815680 26859 raft_consensus.cc:2454] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 256ce64303ec49e1a5cd8e0ffee436b0 in term 1.
I20240502 13:59:54.815806 27386 raft_consensus.cc:2454] T 41e73be38c4348a5bb86da6c0f6c6d22 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 256ce64303ec49e1a5cd8e0ffee436b0 in term 1.
I20240502 13:59:54.816217 27123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e73be38c4348a5bb86da6c0f6c6d22" candidate_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0280081e4ff74896832ec3985c372b3a"
I20240502 13:59:54.816376 27123 raft_consensus.cc:3037] T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.816395 26925 leader_election.cc:302] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 256ce64303ec49e1a5cd8e0ffee436b0, d3e36186faa442b7bd87357e74d47f68, e44f8c16a14a48a38a5a74af2899c217; no voters:
I20240502 13:59:54.816231 27122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95f064e56ee241dba95f9a6cd5984d97" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0280081e4ff74896832ec3985c372b3a"
I20240502 13:59:54.816639 27122 raft_consensus.cc:3037] T 95f064e56ee241dba95f9a6cd5984d97 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.817107 26859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95f064e56ee241dba95f9a6cd5984d97" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e36186faa442b7bd87357e74d47f68"
I20240502 13:59:54.817243 26859 raft_consensus.cc:3037] T 95f064e56ee241dba95f9a6cd5984d97 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.817369 27123 raft_consensus.cc:2454] T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 256ce64303ec49e1a5cd8e0ffee436b0 in term 1.
I20240502 13:59:54.817456 27122 raft_consensus.cc:2454] T 95f064e56ee241dba95f9a6cd5984d97 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 1.
I20240502 13:59:54.817641 27254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41e73be38c4348a5bb86da6c0f6c6d22" candidate_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4fc3c795b1d4505b3a2903532d38b52"
I20240502 13:59:54.817754 27255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a74c0c312a7403d89958a3632cab7d3" candidate_uuid: "0280081e4ff74896832ec3985c372b3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4fc3c795b1d4505b3a2903532d38b52"
I20240502 13:59:54.817759 27254 raft_consensus.cc:3037] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.817881 27255 raft_consensus.cc:3037] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.817898 26858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a74c0c312a7403d89958a3632cab7d3" candidate_uuid: "0280081e4ff74896832ec3985c372b3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e36186faa442b7bd87357e74d47f68"
I20240502 13:59:54.817996 26858 raft_consensus.cc:3037] T 4a74c0c312a7403d89958a3632cab7d3 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.818279 27797 raft_consensus.cc:2781] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:54.818470 27797 raft_consensus.cc:686] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 LEADER]: Becoming Leader. State: Replica: 256ce64303ec49e1a5cd8e0ffee436b0, State: Running, Role: LEADER
I20240502 13:59:54.818770 27387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a74c0c312a7403d89958a3632cab7d3" candidate_uuid: "0280081e4ff74896832ec3985c372b3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44f8c16a14a48a38a5a74af2899c217"
I20240502 13:59:54.818898 27387 raft_consensus.cc:3037] T 4a74c0c312a7403d89958a3632cab7d3 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.818847 27797 consensus_queue.cc:237] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:54.819213 27519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95f064e56ee241dba95f9a6cd5984d97" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b161bd719d4d72baf8da4f40aef136"
I20240502 13:59:54.819330 27519 raft_consensus.cc:3037] T 95f064e56ee241dba95f9a6cd5984d97 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.819389 27386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95f064e56ee241dba95f9a6cd5984d97" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44f8c16a14a48a38a5a74af2899c217"
I20240502 13:59:54.819482 27386 raft_consensus.cc:3037] T 95f064e56ee241dba95f9a6cd5984d97 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:54.821637 26711 catalog_manager.cc:5484] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66). New cstate: current_term: 1 leader_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: HEALTHY } } }
I20240502 13:59:54.909577 27387 raft_consensus.cc:2454] T 4a74c0c312a7403d89958a3632cab7d3 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0280081e4ff74896832ec3985c372b3a in term 1.
I20240502 13:59:54.909577 27255 raft_consensus.cc:2454] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0280081e4ff74896832ec3985c372b3a in term 1.
I20240502 13:59:54.909641 26858 raft_consensus.cc:2454] T 4a74c0c312a7403d89958a3632cab7d3 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0280081e4ff74896832ec3985c372b3a in term 1.
I20240502 13:59:54.909973 27386 raft_consensus.cc:2454] T 95f064e56ee241dba95f9a6cd5984d97 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 1.
I20240502 13:59:54.910161 27056 leader_election.cc:302] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0280081e4ff74896832ec3985c372b3a, 34b161bd719d4d72baf8da4f40aef136, d3e36186faa442b7bd87357e74d47f68; no voters:
I20240502 13:59:54.910367 27254 raft_consensus.cc:2454] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 256ce64303ec49e1a5cd8e0ffee436b0 in term 1.
I20240502 13:59:54.910396 27791 raft_consensus.cc:2781] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:54.910624 27791 raft_consensus.cc:686] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [term 1 LEADER]: Becoming Leader. State: Replica: 0280081e4ff74896832ec3985c372b3a, State: Running, Role: LEADER
I20240502 13:59:54.910669 27519 raft_consensus.cc:2454] T 95f064e56ee241dba95f9a6cd5984d97 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 1.
I20240502 13:59:54.911046 27188 leader_election.cc:302] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0280081e4ff74896832ec3985c372b3a, 34b161bd719d4d72baf8da4f40aef136, b4fc3c795b1d4505b3a2903532d38b52; no voters:
I20240502 13:59:54.911547 27896 raft_consensus.cc:2781] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:54.911633 27896 raft_consensus.cc:686] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 LEADER]: Becoming Leader. State: Replica: b4fc3c795b1d4505b3a2903532d38b52, State: Running, Role: LEADER
I20240502 13:59:54.911792 26859 raft_consensus.cc:2454] T 95f064e56ee241dba95f9a6cd5984d97 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 1.
I20240502 13:59:54.911784 27896 consensus_queue.cc:237] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.911027 27791 consensus_queue.cc:237] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:54.915011 26711 catalog_manager.cc:5484] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68). New cstate: current_term: 1 leader_uuid: "b4fc3c795b1d4505b3a2903532d38b52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: HEALTHY } } }
I20240502 13:59:54.916278 26714 catalog_manager.cc:5484] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0280081e4ff74896832ec3985c372b3a (127.24.66.67). New cstate: current_term: 1 leader_uuid: "0280081e4ff74896832ec3985c372b3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: UNKNOWN } } }
W20240502 13:59:55.085608 27941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:55.085889 27941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:55.089252 27941 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240502 13:59:55.182653 27122 raft_consensus.cc:1261] T 95f064e56ee241dba95f9a6cd5984d97 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:55.182731 26858 raft_consensus.cc:1261] T 95f064e56ee241dba95f9a6cd5984d97 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:55.183398 27519 raft_consensus.cc:1261] T 95f064e56ee241dba95f9a6cd5984d97 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:55.183662 27814 consensus_queue.cc:1035] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:55.183845 27386 raft_consensus.cc:1261] T 95f064e56ee241dba95f9a6cd5984d97 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:55.183969 27814 consensus_queue.cc:1035] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:55.184235 27814 consensus_queue.cc:1035] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:55.184527 27814 consensus_queue.cc:1035] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:55.190034 27254 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:55.190475 27791 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:55.193152 27387 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:55.193470 26859 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:55.194218 27791 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:55.194350 27938 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:55.195189 27518 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:55.197422 27385 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:55.197866 27797 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:55.197935 27791 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:55.198035 26857 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:55.198504 27797 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:55.200683 27123 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:55.201057 27797 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:55.204129 27255 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:55.204547 27797 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 13:59:55.241762 27979 log.cc:924] Time spent T 95f064e56ee241dba95f9a6cd5984d97 P d3e36186faa442b7bd87357e74d47f68: Append to log took a long time: real 0.053s user 0.000s sys 0.000s
W20240502 13:59:55.242674 27940 log.cc:924] Time spent T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a: Append to log took a long time: real 0.053s user 0.000s sys 0.000s
W20240502 13:59:55.251526 27977 log.cc:924] Time spent T 95f064e56ee241dba95f9a6cd5984d97 P 0280081e4ff74896832ec3985c372b3a: Append to log took a long time: real 0.056s user 0.000s sys 0.000s
W20240502 13:59:55.264405 27998 log.cc:924] Time spent T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52: Append to log took a long time: real 0.058s user 0.000s sys 0.000s
W20240502 13:59:55.265031 27995 log.cc:924] Time spent T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136: Append to log took a long time: real 0.053s user 0.000s sys 0.000s
W20240502 13:59:55.275023 27997 log.cc:924] Time spent T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a: Append to log took a long time: real 0.062s user 0.000s sys 0.000s
INSERT report
rows total: 2000
time total: 203.35 ms
time per row: 0.101675 ms
I20240502 13:59:55.337667 26713 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50400:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 13:59:55.343223 27102 tablet_service.cc:1467] Processing CreateTablet for tablet df55bedff83e4a5d822547e0fa1c2f87 (DEFAULT_TABLE table=rebalance_test_table_2 [id=72ce8466a12e4262b615aa4cc05d3b93]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.343564 27102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df55bedff83e4a5d822547e0fa1c2f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.343535 26837 tablet_service.cc:1467] Processing CreateTablet for tablet df55bedff83e4a5d822547e0fa1c2f87 (DEFAULT_TABLE table=rebalance_test_table_2 [id=72ce8466a12e4262b615aa4cc05d3b93]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.343854 26837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df55bedff83e4a5d822547e0fa1c2f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.344936 27498 tablet_service.cc:1467] Processing CreateTablet for tablet df55bedff83e4a5d822547e0fa1c2f87 (DEFAULT_TABLE table=rebalance_test_table_2 [id=72ce8466a12e4262b615aa4cc05d3b93]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.345073 27367 tablet_service.cc:1467] Processing CreateTablet for tablet 945d054a56ed4043b7b49d14ed917532 (DEFAULT_TABLE table=rebalance_test_table_2 [id=72ce8466a12e4262b615aa4cc05d3b93]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.345203 27498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df55bedff83e4a5d822547e0fa1c2f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.345348 27367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 945d054a56ed4043b7b49d14ed917532. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.347299 28025 tablet_bootstrap.cc:492] T df55bedff83e4a5d822547e0fa1c2f87 P 34b161bd719d4d72baf8da4f40aef136: Bootstrap starting.
I20240502 13:59:55.347506 27497 tablet_service.cc:1467] Processing CreateTablet for tablet a734b1d6d928408cac2a9e1b263f2697 (DEFAULT_TABLE table=rebalance_test_table_2 [id=72ce8466a12e4262b615aa4cc05d3b93]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.347669 27499 tablet_service.cc:1467] Processing CreateTablet for tablet 945d054a56ed4043b7b49d14ed917532 (DEFAULT_TABLE table=rebalance_test_table_2 [id=72ce8466a12e4262b615aa4cc05d3b93]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.348204 28025 tablet_bootstrap.cc:654] T df55bedff83e4a5d822547e0fa1c2f87 P 34b161bd719d4d72baf8da4f40aef136: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.348615 26969 tablet_service.cc:1467] Processing CreateTablet for tablet df55bedff83e4a5d822547e0fa1c2f87 (DEFAULT_TABLE table=rebalance_test_table_2 [id=72ce8466a12e4262b615aa4cc05d3b93]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.348682 26971 tablet_service.cc:1467] Processing CreateTablet for tablet 945d054a56ed4043b7b49d14ed917532 (DEFAULT_TABLE table=rebalance_test_table_2 [id=72ce8466a12e4262b615aa4cc05d3b93]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.348919 26969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df55bedff83e4a5d822547e0fa1c2f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.349043 26970 tablet_service.cc:1467] Processing CreateTablet for tablet a734b1d6d928408cac2a9e1b263f2697 (DEFAULT_TABLE table=rebalance_test_table_2 [id=72ce8466a12e4262b615aa4cc05d3b93]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.349362 26970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a734b1d6d928408cac2a9e1b263f2697. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.349409 26838 tablet_service.cc:1467] Processing CreateTablet for tablet 945d054a56ed4043b7b49d14ed917532 (DEFAULT_TABLE table=rebalance_test_table_2 [id=72ce8466a12e4262b615aa4cc05d3b93]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.349649 26838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 945d054a56ed4043b7b49d14ed917532. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.350992 28028 tablet_bootstrap.cc:492] T a734b1d6d928408cac2a9e1b263f2697 P 256ce64303ec49e1a5cd8e0ffee436b0: Bootstrap starting.
I20240502 13:59:55.351274 26971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 945d054a56ed4043b7b49d14ed917532. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.351891 28028 tablet_bootstrap.cc:654] T a734b1d6d928408cac2a9e1b263f2697 P 256ce64303ec49e1a5cd8e0ffee436b0: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.347734 27497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a734b1d6d928408cac2a9e1b263f2697. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.352838 27499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 945d054a56ed4043b7b49d14ed917532. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.353773 27233 tablet_service.cc:1467] Processing CreateTablet for tablet df55bedff83e4a5d822547e0fa1c2f87 (DEFAULT_TABLE table=rebalance_test_table_2 [id=72ce8466a12e4262b615aa4cc05d3b93]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.354063 27233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df55bedff83e4a5d822547e0fa1c2f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.354182 27102 tablet_service.cc:1467] Processing CreateTablet for tablet 945d054a56ed4043b7b49d14ed917532 (DEFAULT_TABLE table=rebalance_test_table_2 [id=72ce8466a12e4262b615aa4cc05d3b93]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.354403 27102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 945d054a56ed4043b7b49d14ed917532. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.356201 28031 tablet_bootstrap.cc:492] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52: Bootstrap starting.
I20240502 13:59:55.353773 27234 tablet_service.cc:1467] Processing CreateTablet for tablet a734b1d6d928408cac2a9e1b263f2697 (DEFAULT_TABLE table=rebalance_test_table_2 [id=72ce8466a12e4262b615aa4cc05d3b93]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.356523 27234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a734b1d6d928408cac2a9e1b263f2697. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.356581 28026 tablet_bootstrap.cc:492] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217: Bootstrap starting.
I20240502 13:59:55.356698 28028 tablet_bootstrap.cc:492] T a734b1d6d928408cac2a9e1b263f2697 P 256ce64303ec49e1a5cd8e0ffee436b0: No bootstrap required, opened a new log
I20240502 13:59:55.356992 28028 ts_tablet_manager.cc:1367] T a734b1d6d928408cac2a9e1b263f2697 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 13:59:55.357339 28031 tablet_bootstrap.cc:654] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.357417 28026 tablet_bootstrap.cc:654] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.357555 28028 raft_consensus.cc:348] T a734b1d6d928408cac2a9e1b263f2697 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:55.357710 28028 raft_consensus.cc:374] T a734b1d6d928408cac2a9e1b263f2697 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.357753 28028 raft_consensus.cc:729] T a734b1d6d928408cac2a9e1b263f2697 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 256ce64303ec49e1a5cd8e0ffee436b0, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.357903 28028 consensus_queue.cc:260] T a734b1d6d928408cac2a9e1b263f2697 P 256ce64303ec49e1a5cd8e0ffee436b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:55.358114 28030 tablet_bootstrap.cc:492] T df55bedff83e4a5d822547e0fa1c2f87 P 0280081e4ff74896832ec3985c372b3a: Bootstrap starting.
I20240502 13:59:55.358224 28028 ts_tablet_manager.cc:1398] T a734b1d6d928408cac2a9e1b263f2697 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:55.358839 28028 tablet_bootstrap.cc:492] T df55bedff83e4a5d822547e0fa1c2f87 P 256ce64303ec49e1a5cd8e0ffee436b0: Bootstrap starting.
I20240502 13:59:55.359054 28030 tablet_bootstrap.cc:654] T df55bedff83e4a5d822547e0fa1c2f87 P 0280081e4ff74896832ec3985c372b3a: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.359652 26838 tablet_service.cc:1467] Processing CreateTablet for tablet a734b1d6d928408cac2a9e1b263f2697 (DEFAULT_TABLE table=rebalance_test_table_2 [id=72ce8466a12e4262b615aa4cc05d3b93]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.359740 28028 tablet_bootstrap.cc:654] T df55bedff83e4a5d822547e0fa1c2f87 P 256ce64303ec49e1a5cd8e0ffee436b0: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.359885 26838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a734b1d6d928408cac2a9e1b263f2697. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.360603 27367 tablet_service.cc:1467] Processing CreateTablet for tablet a734b1d6d928408cac2a9e1b263f2697 (DEFAULT_TABLE table=rebalance_test_table_2 [id=72ce8466a12e4262b615aa4cc05d3b93]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.360872 27367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a734b1d6d928408cac2a9e1b263f2697. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.361060 28024 tablet_bootstrap.cc:492] T df55bedff83e4a5d822547e0fa1c2f87 P d3e36186faa442b7bd87357e74d47f68: Bootstrap starting.
I20240502 13:59:55.362439 28026 tablet_bootstrap.cc:492] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217: No bootstrap required, opened a new log
I20240502 13:59:55.362532 28026 ts_tablet_manager.cc:1367] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 13:59:55.362933 28026 raft_consensus.cc:348] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } }
I20240502 13:59:55.363051 28026 raft_consensus.cc:374] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.363090 28026 raft_consensus.cc:729] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44f8c16a14a48a38a5a74af2899c217, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.363118 28024 tablet_bootstrap.cc:654] T df55bedff83e4a5d822547e0fa1c2f87 P d3e36186faa442b7bd87357e74d47f68: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.363220 28026 consensus_queue.cc:260] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } }
I20240502 13:59:55.363353 28028 tablet_bootstrap.cc:492] T df55bedff83e4a5d822547e0fa1c2f87 P 256ce64303ec49e1a5cd8e0ffee436b0: No bootstrap required, opened a new log
I20240502 13:59:55.363456 28028 ts_tablet_manager.cc:1367] T df55bedff83e4a5d822547e0fa1c2f87 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 13:59:55.363576 28026 ts_tablet_manager.cc:1398] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:55.363711 28026 tablet_bootstrap.cc:492] T a734b1d6d928408cac2a9e1b263f2697 P e44f8c16a14a48a38a5a74af2899c217: Bootstrap starting.
I20240502 13:59:55.363926 28028 raft_consensus.cc:348] T df55bedff83e4a5d822547e0fa1c2f87 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:55.364046 28028 raft_consensus.cc:374] T df55bedff83e4a5d822547e0fa1c2f87 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.364095 28028 raft_consensus.cc:729] T df55bedff83e4a5d822547e0fa1c2f87 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 256ce64303ec49e1a5cd8e0ffee436b0, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.364221 28028 consensus_queue.cc:260] T df55bedff83e4a5d822547e0fa1c2f87 P 256ce64303ec49e1a5cd8e0ffee436b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:55.364547 28028 ts_tablet_manager.cc:1398] T df55bedff83e4a5d822547e0fa1c2f87 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 13:59:55.364722 28026 tablet_bootstrap.cc:654] T a734b1d6d928408cac2a9e1b263f2697 P e44f8c16a14a48a38a5a74af2899c217: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.365480 28025 tablet_bootstrap.cc:492] T df55bedff83e4a5d822547e0fa1c2f87 P 34b161bd719d4d72baf8da4f40aef136: No bootstrap required, opened a new log
I20240502 13:59:55.365494 28031 tablet_bootstrap.cc:492] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52: No bootstrap required, opened a new log
I20240502 13:59:55.365574 28025 ts_tablet_manager.cc:1367] T df55bedff83e4a5d822547e0fa1c2f87 P 34b161bd719d4d72baf8da4f40aef136: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240502 13:59:55.365583 28031 ts_tablet_manager.cc:1367] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 13:59:55.365926 28028 tablet_bootstrap.cc:492] T 945d054a56ed4043b7b49d14ed917532 P 256ce64303ec49e1a5cd8e0ffee436b0: Bootstrap starting.
I20240502 13:59:55.366011 28025 raft_consensus.cc:348] T df55bedff83e4a5d822547e0fa1c2f87 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:55.366065 28031 raft_consensus.cc:348] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:55.366140 28025 raft_consensus.cc:374] T df55bedff83e4a5d822547e0fa1c2f87 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.366178 28025 raft_consensus.cc:729] T df55bedff83e4a5d822547e0fa1c2f87 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34b161bd719d4d72baf8da4f40aef136, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.366182 28031 raft_consensus.cc:374] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.366220 28031 raft_consensus.cc:729] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4fc3c795b1d4505b3a2903532d38b52, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.366344 28025 consensus_queue.cc:260] T df55bedff83e4a5d822547e0fa1c2f87 P 34b161bd719d4d72baf8da4f40aef136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:55.366349 28031 consensus_queue.cc:260] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:55.366623 28031 ts_tablet_manager.cc:1398] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:55.366811 28031 tablet_bootstrap.cc:492] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52: Bootstrap starting.
I20240502 13:59:55.367159 28025 ts_tablet_manager.cc:1398] T df55bedff83e4a5d822547e0fa1c2f87 P 34b161bd719d4d72baf8da4f40aef136: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:55.367326 28025 tablet_bootstrap.cc:492] T 945d054a56ed4043b7b49d14ed917532 P 34b161bd719d4d72baf8da4f40aef136: Bootstrap starting.
I20240502 13:59:55.367503 27974 raft_consensus.cc:482] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:55.367596 27974 raft_consensus.cc:504] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:55.367671 28031 tablet_bootstrap.cc:654] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.368065 27974 leader_election.cc:288] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733), 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217), d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841), 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473)
I20240502 13:59:55.368556 26989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df55bedff83e4a5d822547e0fa1c2f87" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" is_pre_election: true
I20240502 13:59:55.368681 26989 raft_consensus.cc:2454] T df55bedff83e4a5d822547e0fa1c2f87 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 0.
I20240502 13:59:55.368736 27123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df55bedff83e4a5d822547e0fa1c2f87" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0280081e4ff74896832ec3985c372b3a" is_pre_election: true
I20240502 13:59:55.369092 28028 tablet_bootstrap.cc:654] T 945d054a56ed4043b7b49d14ed917532 P 256ce64303ec49e1a5cd8e0ffee436b0: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.369139 27518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df55bedff83e4a5d822547e0fa1c2f87" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b161bd719d4d72baf8da4f40aef136" is_pre_election: true
I20240502 13:59:55.369246 27518 raft_consensus.cc:2454] T df55bedff83e4a5d822547e0fa1c2f87 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 0.
W20240502 13:59:55.369239 27191 leader_election.cc:341] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733): Illegal state: must be running to vote when last-logged opid is not known
I20240502 13:59:55.369624 26858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df55bedff83e4a5d822547e0fa1c2f87" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e36186faa442b7bd87357e74d47f68" is_pre_election: true
I20240502 13:59:55.369866 27188 leader_election.cc:302] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 5 voters: 3 yes votes; 1 no votes. yes voters: 256ce64303ec49e1a5cd8e0ffee436b0, 34b161bd719d4d72baf8da4f40aef136, b4fc3c795b1d4505b3a2903532d38b52; no voters: 0280081e4ff74896832ec3985c372b3a
I20240502 13:59:55.370013 27814 raft_consensus.cc:2781] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:55.370083 27814 raft_consensus.cc:482] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:55.370129 27814 raft_consensus.cc:3037] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Advancing to term 1
W20240502 13:59:55.370139 27188 leader_election.cc:341] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841): Illegal state: must be running to vote when last-logged opid is not known
I20240502 13:59:55.371701 28031 tablet_bootstrap.cc:492] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52: No bootstrap required, opened a new log
I20240502 13:59:55.371807 28031 ts_tablet_manager.cc:1367] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 13:59:55.372270 28025 tablet_bootstrap.cc:654] T 945d054a56ed4043b7b49d14ed917532 P 34b161bd719d4d72baf8da4f40aef136: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.372268 28031 raft_consensus.cc:348] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:55.372428 28031 raft_consensus.cc:374] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.372464 28031 raft_consensus.cc:729] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4fc3c795b1d4505b3a2903532d38b52, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.372602 28031 consensus_queue.cc:260] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:55.372673 28024 tablet_bootstrap.cc:492] T df55bedff83e4a5d822547e0fa1c2f87 P d3e36186faa442b7bd87357e74d47f68: No bootstrap required, opened a new log
I20240502 13:59:55.372757 28024 ts_tablet_manager.cc:1367] T df55bedff83e4a5d822547e0fa1c2f87 P d3e36186faa442b7bd87357e74d47f68: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 13:59:55.372897 28031 ts_tablet_manager.cc:1398] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:55.373231 28024 raft_consensus.cc:348] T df55bedff83e4a5d822547e0fa1c2f87 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:55.373369 28024 raft_consensus.cc:374] T df55bedff83e4a5d822547e0fa1c2f87 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.373411 28024 raft_consensus.cc:729] T df55bedff83e4a5d822547e0fa1c2f87 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3e36186faa442b7bd87357e74d47f68, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.373584 28024 consensus_queue.cc:260] T df55bedff83e4a5d822547e0fa1c2f87 P d3e36186faa442b7bd87357e74d47f68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:55.373790 28025 tablet_bootstrap.cc:492] T 945d054a56ed4043b7b49d14ed917532 P 34b161bd719d4d72baf8da4f40aef136: No bootstrap required, opened a new log
I20240502 13:59:55.373883 28025 ts_tablet_manager.cc:1367] T 945d054a56ed4043b7b49d14ed917532 P 34b161bd719d4d72baf8da4f40aef136: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 13:59:55.374042 28030 tablet_bootstrap.cc:492] T df55bedff83e4a5d822547e0fa1c2f87 P 0280081e4ff74896832ec3985c372b3a: No bootstrap required, opened a new log
I20240502 13:59:55.374109 28024 ts_tablet_manager.cc:1398] T df55bedff83e4a5d822547e0fa1c2f87 P d3e36186faa442b7bd87357e74d47f68: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:55.374142 28028 tablet_bootstrap.cc:492] T 945d054a56ed4043b7b49d14ed917532 P 256ce64303ec49e1a5cd8e0ffee436b0: No bootstrap required, opened a new log
I20240502 13:59:55.374212 28030 ts_tablet_manager.cc:1367] T df55bedff83e4a5d822547e0fa1c2f87 P 0280081e4ff74896832ec3985c372b3a: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 13:59:55.374228 28028 ts_tablet_manager.cc:1367] T 945d054a56ed4043b7b49d14ed917532 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20240502 13:59:55.374284 28024 tablet_bootstrap.cc:492] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68: Bootstrap starting.
I20240502 13:59:55.374339 28025 raft_consensus.cc:348] T 945d054a56ed4043b7b49d14ed917532 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } }
I20240502 13:59:55.374459 28025 raft_consensus.cc:374] T 945d054a56ed4043b7b49d14ed917532 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.374490 28025 raft_consensus.cc:729] T 945d054a56ed4043b7b49d14ed917532 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34b161bd719d4d72baf8da4f40aef136, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.374599 28025 consensus_queue.cc:260] T 945d054a56ed4043b7b49d14ed917532 P 34b161bd719d4d72baf8da4f40aef136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } }
I20240502 13:59:55.374691 28028 raft_consensus.cc:348] T 945d054a56ed4043b7b49d14ed917532 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } }
I20240502 13:59:55.374809 28028 raft_consensus.cc:374] T 945d054a56ed4043b7b49d14ed917532 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.374850 28028 raft_consensus.cc:729] T 945d054a56ed4043b7b49d14ed917532 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 256ce64303ec49e1a5cd8e0ffee436b0, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.374889 28025 ts_tablet_manager.cc:1398] T 945d054a56ed4043b7b49d14ed917532 P 34b161bd719d4d72baf8da4f40aef136: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:55.374991 28028 consensus_queue.cc:260] T 945d054a56ed4043b7b49d14ed917532 P 256ce64303ec49e1a5cd8e0ffee436b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } }
I20240502 13:59:55.375064 28025 tablet_bootstrap.cc:492] T a734b1d6d928408cac2a9e1b263f2697 P 34b161bd719d4d72baf8da4f40aef136: Bootstrap starting.
I20240502 13:59:55.375288 28028 ts_tablet_manager.cc:1398] T 945d054a56ed4043b7b49d14ed917532 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:55.375523 28030 raft_consensus.cc:348] T df55bedff83e4a5d822547e0fa1c2f87 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:55.375708 28030 raft_consensus.cc:374] T df55bedff83e4a5d822547e0fa1c2f87 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.375787 28030 raft_consensus.cc:729] T df55bedff83e4a5d822547e0fa1c2f87 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0280081e4ff74896832ec3985c372b3a, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.375928 28025 tablet_bootstrap.cc:654] T a734b1d6d928408cac2a9e1b263f2697 P 34b161bd719d4d72baf8da4f40aef136: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.376061 28024 tablet_bootstrap.cc:654] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.376041 28030 consensus_queue.cc:260] T df55bedff83e4a5d822547e0fa1c2f87 P 0280081e4ff74896832ec3985c372b3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:55.377044 28030 ts_tablet_manager.cc:1398] T df55bedff83e4a5d822547e0fa1c2f87 P 0280081e4ff74896832ec3985c372b3a: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 13:59:55.377265 28030 tablet_bootstrap.cc:492] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a: Bootstrap starting.
I20240502 13:59:55.377595 28025 tablet_bootstrap.cc:492] T a734b1d6d928408cac2a9e1b263f2697 P 34b161bd719d4d72baf8da4f40aef136: No bootstrap required, opened a new log
I20240502 13:59:55.377681 28025 ts_tablet_manager.cc:1367] T a734b1d6d928408cac2a9e1b263f2697 P 34b161bd719d4d72baf8da4f40aef136: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 13:59:55.378094 28025 raft_consensus.cc:348] T a734b1d6d928408cac2a9e1b263f2697 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:55.378204 28025 raft_consensus.cc:374] T a734b1d6d928408cac2a9e1b263f2697 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.378258 28025 raft_consensus.cc:729] T a734b1d6d928408cac2a9e1b263f2697 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34b161bd719d4d72baf8da4f40aef136, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.378425 28026 tablet_bootstrap.cc:492] T a734b1d6d928408cac2a9e1b263f2697 P e44f8c16a14a48a38a5a74af2899c217: No bootstrap required, opened a new log
I20240502 13:59:55.378383 28025 consensus_queue.cc:260] T a734b1d6d928408cac2a9e1b263f2697 P 34b161bd719d4d72baf8da4f40aef136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:55.378532 28026 ts_tablet_manager.cc:1367] T a734b1d6d928408cac2a9e1b263f2697 P e44f8c16a14a48a38a5a74af2899c217: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20240502 13:59:55.378675 28025 ts_tablet_manager.cc:1398] T a734b1d6d928408cac2a9e1b263f2697 P 34b161bd719d4d72baf8da4f40aef136: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:55.378830 28030 tablet_bootstrap.cc:654] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.379042 28026 raft_consensus.cc:348] T a734b1d6d928408cac2a9e1b263f2697 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:55.379163 28026 raft_consensus.cc:374] T a734b1d6d928408cac2a9e1b263f2697 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.379201 28026 raft_consensus.cc:729] T a734b1d6d928408cac2a9e1b263f2697 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44f8c16a14a48a38a5a74af2899c217, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.379323 28026 consensus_queue.cc:260] T a734b1d6d928408cac2a9e1b263f2697 P e44f8c16a14a48a38a5a74af2899c217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:55.379626 28026 ts_tablet_manager.cc:1398] T a734b1d6d928408cac2a9e1b263f2697 P e44f8c16a14a48a38a5a74af2899c217: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:55.380393 28024 tablet_bootstrap.cc:492] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68: No bootstrap required, opened a new log
I20240502 13:59:55.380492 28024 ts_tablet_manager.cc:1367] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 13:59:55.380930 28024 raft_consensus.cc:348] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } }
I20240502 13:59:55.381049 28024 raft_consensus.cc:374] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.381086 28024 raft_consensus.cc:729] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3e36186faa442b7bd87357e74d47f68, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.381209 28024 consensus_queue.cc:260] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } }
I20240502 13:59:55.381474 28024 ts_tablet_manager.cc:1398] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:55.381651 28024 tablet_bootstrap.cc:492] T a734b1d6d928408cac2a9e1b263f2697 P d3e36186faa442b7bd87357e74d47f68: Bootstrap starting.
I20240502 13:59:55.382488 28030 tablet_bootstrap.cc:492] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a: No bootstrap required, opened a new log
I20240502 13:59:55.382531 28024 tablet_bootstrap.cc:654] T a734b1d6d928408cac2a9e1b263f2697 P d3e36186faa442b7bd87357e74d47f68: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.382584 28030 ts_tablet_manager.cc:1367] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 13:59:55.382961 28030 raft_consensus.cc:348] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } }
I20240502 13:59:55.383061 28030 raft_consensus.cc:374] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.383095 28030 raft_consensus.cc:729] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0280081e4ff74896832ec3985c372b3a, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.383244 28030 consensus_queue.cc:260] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } }
I20240502 13:59:55.383915 28024 tablet_bootstrap.cc:492] T a734b1d6d928408cac2a9e1b263f2697 P d3e36186faa442b7bd87357e74d47f68: No bootstrap required, opened a new log
I20240502 13:59:55.383998 28024 ts_tablet_manager.cc:1367] T a734b1d6d928408cac2a9e1b263f2697 P d3e36186faa442b7bd87357e74d47f68: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:55.384102 28030 ts_tablet_manager.cc:1398] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:55.384464 28024 raft_consensus.cc:348] T a734b1d6d928408cac2a9e1b263f2697 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:55.384572 28024 raft_consensus.cc:374] T a734b1d6d928408cac2a9e1b263f2697 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.384613 28024 raft_consensus.cc:729] T a734b1d6d928408cac2a9e1b263f2697 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3e36186faa442b7bd87357e74d47f68, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.384734 28024 consensus_queue.cc:260] T a734b1d6d928408cac2a9e1b263f2697 P d3e36186faa442b7bd87357e74d47f68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:55.384984 28024 ts_tablet_manager.cc:1398] T a734b1d6d928408cac2a9e1b263f2697 P d3e36186faa442b7bd87357e74d47f68: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:55.391860 27974 raft_consensus.cc:482] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:55.391964 27974 raft_consensus.cc:504] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:55.392521 27974 leader_election.cc:288] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217), e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107), 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473), d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841)
I20240502 13:59:55.392743 27518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a734b1d6d928408cac2a9e1b263f2697" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b161bd719d4d72baf8da4f40aef136" is_pre_election: true
I20240502 13:59:55.392820 26858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a734b1d6d928408cac2a9e1b263f2697" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e36186faa442b7bd87357e74d47f68" is_pre_election: true
I20240502 13:59:55.392832 27385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a734b1d6d928408cac2a9e1b263f2697" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44f8c16a14a48a38a5a74af2899c217" is_pre_election: true
I20240502 13:59:55.392874 27518 raft_consensus.cc:2454] T a734b1d6d928408cac2a9e1b263f2697 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 0.
I20240502 13:59:55.392865 26989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a734b1d6d928408cac2a9e1b263f2697" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" is_pre_election: true
I20240502 13:59:55.392940 26858 raft_consensus.cc:2454] T a734b1d6d928408cac2a9e1b263f2697 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 0.
I20240502 13:59:55.392969 27385 raft_consensus.cc:2454] T a734b1d6d928408cac2a9e1b263f2697 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 0.
I20240502 13:59:55.392979 26989 raft_consensus.cc:2454] T a734b1d6d928408cac2a9e1b263f2697 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 0.
I20240502 13:59:55.393222 27188 leader_election.cc:302] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 34b161bd719d4d72baf8da4f40aef136, b4fc3c795b1d4505b3a2903532d38b52, d3e36186faa442b7bd87357e74d47f68; no voters:
I20240502 13:59:55.393404 27974 raft_consensus.cc:2781] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:55.393491 27974 raft_consensus.cc:482] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:55.393538 27974 raft_consensus.cc:3037] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:55.432932 28005 raft_consensus.cc:482] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:55.433074 28005 raft_consensus.cc:504] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } }
I20240502 13:59:55.433805 28005 leader_election.cc:288] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473), d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841), 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217), 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733)
I20240502 13:59:55.434201 26989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "945d054a56ed4043b7b49d14ed917532" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" is_pre_election: true
I20240502 13:59:55.434353 26989 raft_consensus.cc:2454] T 945d054a56ed4043b7b49d14ed917532 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 0.
I20240502 13:59:55.434451 27518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "945d054a56ed4043b7b49d14ed917532" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b161bd719d4d72baf8da4f40aef136" is_pre_election: true
I20240502 13:59:55.434587 27518 raft_consensus.cc:2454] T 945d054a56ed4043b7b49d14ed917532 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 0.
I20240502 13:59:55.434878 27320 leader_election.cc:302] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 256ce64303ec49e1a5cd8e0ffee436b0, 34b161bd719d4d72baf8da4f40aef136, e44f8c16a14a48a38a5a74af2899c217; no voters:
I20240502 13:59:55.435027 28005 raft_consensus.cc:2781] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:55.435088 28005 raft_consensus.cc:482] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:55.435134 28005 raft_consensus.cc:3037] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:55.438161 26858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "945d054a56ed4043b7b49d14ed917532" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e36186faa442b7bd87357e74d47f68" is_pre_election: true
I20240502 13:59:55.438309 26858 raft_consensus.cc:2454] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 0.
I20240502 13:59:55.438795 27123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "945d054a56ed4043b7b49d14ed917532" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0280081e4ff74896832ec3985c372b3a" is_pre_election: true
I20240502 13:59:55.438911 27123 raft_consensus.cc:2454] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 0.
I20240502 13:59:55.446727 28005 raft_consensus.cc:504] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } }
I20240502 13:59:55.446725 27814 raft_consensus.cc:504] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:55.447315 28005 leader_election.cc:288] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [CANDIDATE]: Term 1 election: Requested vote from peers 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473), d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841), 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217), 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733)
I20240502 13:59:55.447415 27814 leader_election.cc:288] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52 [CANDIDATE]: Term 1 election: Requested vote from peers 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733), 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217), d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841), 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473)
I20240502 13:59:55.447547 27974 raft_consensus.cc:504] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:55.448017 27974 leader_election.cc:288] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [CANDIDATE]: Term 1 election: Requested vote from peers 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217), e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107), 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473), d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841)
I20240502 13:59:55.448239 27518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "945d054a56ed4043b7b49d14ed917532" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b161bd719d4d72baf8da4f40aef136"
I20240502 13:59:55.448318 26989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "945d054a56ed4043b7b49d14ed917532" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0"
I20240502 13:59:55.448360 27123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df55bedff83e4a5d822547e0fa1c2f87" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0280081e4ff74896832ec3985c372b3a"
I20240502 13:59:55.448449 26989 raft_consensus.cc:3037] T 945d054a56ed4043b7b49d14ed917532 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:55.448422 27518 raft_consensus.cc:3037] T 945d054a56ed4043b7b49d14ed917532 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:55.448487 27123 raft_consensus.cc:3037] T df55bedff83e4a5d822547e0fa1c2f87 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:55.449481 26989 raft_consensus.cc:2454] T 945d054a56ed4043b7b49d14ed917532 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 1.
I20240502 13:59:55.449486 27518 raft_consensus.cc:2454] T 945d054a56ed4043b7b49d14ed917532 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 1.
I20240502 13:59:55.449640 27123 raft_consensus.cc:2454] T df55bedff83e4a5d822547e0fa1c2f87 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 1.
I20240502 13:59:55.449849 27320 leader_election.cc:302] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 256ce64303ec49e1a5cd8e0ffee436b0, 34b161bd719d4d72baf8da4f40aef136, e44f8c16a14a48a38a5a74af2899c217; no voters:
I20240502 13:59:55.450042 28005 raft_consensus.cc:2781] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:55.450116 28005 raft_consensus.cc:686] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [term 1 LEADER]: Becoming Leader. State: Replica: e44f8c16a14a48a38a5a74af2899c217, State: Running, Role: LEADER
I20240502 13:59:55.450284 28005 consensus_queue.cc:237] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } }
I20240502 13:59:55.450554 26858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "945d054a56ed4043b7b49d14ed917532" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e36186faa442b7bd87357e74d47f68"
I20240502 13:59:55.450688 26858 raft_consensus.cc:3037] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:55.450954 27519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a734b1d6d928408cac2a9e1b263f2697" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b161bd719d4d72baf8da4f40aef136"
I20240502 13:59:55.451057 27519 raft_consensus.cc:3037] T a734b1d6d928408cac2a9e1b263f2697 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:55.451172 26857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df55bedff83e4a5d822547e0fa1c2f87" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e36186faa442b7bd87357e74d47f68"
I20240502 13:59:55.451269 26857 raft_consensus.cc:3037] T df55bedff83e4a5d822547e0fa1c2f87 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:55.451786 26859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a734b1d6d928408cac2a9e1b263f2697" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e36186faa442b7bd87357e74d47f68"
I20240502 13:59:55.451876 26859 raft_consensus.cc:3037] T a734b1d6d928408cac2a9e1b263f2697 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:55.451998 27518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df55bedff83e4a5d822547e0fa1c2f87" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b161bd719d4d72baf8da4f40aef136"
I20240502 13:59:55.452092 27518 raft_consensus.cc:3037] T df55bedff83e4a5d822547e0fa1c2f87 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:55.452383 26991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a734b1d6d928408cac2a9e1b263f2697" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0"
I20240502 13:59:55.452529 26991 raft_consensus.cc:3037] T a734b1d6d928408cac2a9e1b263f2697 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:55.452656 27385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a734b1d6d928408cac2a9e1b263f2697" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44f8c16a14a48a38a5a74af2899c217"
I20240502 13:59:55.452772 27385 raft_consensus.cc:3037] T a734b1d6d928408cac2a9e1b263f2697 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:55.453711 26989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df55bedff83e4a5d822547e0fa1c2f87" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0"
I20240502 13:59:55.453815 26989 raft_consensus.cc:3037] T df55bedff83e4a5d822547e0fa1c2f87 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:55.454154 27123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "945d054a56ed4043b7b49d14ed917532" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0280081e4ff74896832ec3985c372b3a"
I20240502 13:59:55.454273 27123 raft_consensus.cc:3037] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:55.455147 26713 catalog_manager.cc:5484] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 reported cstate change: term changed from 0 to 1, leader changed from <none> to e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69). New cstate: current_term: 1 leader_uuid: "e44f8c16a14a48a38a5a74af2899c217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: UNKNOWN } } }
I20240502 13:59:55.537981 26858 raft_consensus.cc:2454] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 1.
I20240502 13:59:55.537981 27519 raft_consensus.cc:2454] T a734b1d6d928408cac2a9e1b263f2697 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 1.
I20240502 13:59:55.537981 26991 raft_consensus.cc:2454] T a734b1d6d928408cac2a9e1b263f2697 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 1.
I20240502 13:59:55.538954 26857 raft_consensus.cc:2454] T df55bedff83e4a5d822547e0fa1c2f87 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 1.
I20240502 13:59:55.539120 27518 raft_consensus.cc:2454] T df55bedff83e4a5d822547e0fa1c2f87 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 1.
I20240502 13:59:55.539120 26859 raft_consensus.cc:2454] T a734b1d6d928408cac2a9e1b263f2697 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 1.
I20240502 13:59:55.539510 27188 leader_election.cc:302] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0280081e4ff74896832ec3985c372b3a, b4fc3c795b1d4505b3a2903532d38b52, d3e36186faa442b7bd87357e74d47f68; no voters:
I20240502 13:59:55.539623 27123 raft_consensus.cc:2454] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 1.
I20240502 13:59:55.539731 27188 leader_election.cc:302] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 34b161bd719d4d72baf8da4f40aef136, b4fc3c795b1d4505b3a2903532d38b52, d3e36186faa442b7bd87357e74d47f68; no voters:
I20240502 13:59:55.540031 27974 raft_consensus.cc:2781] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:55.540098 27814 raft_consensus.cc:2781] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:55.540138 27974 raft_consensus.cc:686] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 LEADER]: Becoming Leader. State: Replica: b4fc3c795b1d4505b3a2903532d38b52, State: Running, Role: LEADER
I20240502 13:59:55.540151 26989 raft_consensus.cc:2454] T df55bedff83e4a5d822547e0fa1c2f87 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 1.
I20240502 13:59:55.540302 27974 consensus_queue.cc:237] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:55.540309 27814 raft_consensus.cc:686] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 LEADER]: Becoming Leader. State: Replica: b4fc3c795b1d4505b3a2903532d38b52, State: Running, Role: LEADER
I20240502 13:59:55.540872 27814 consensus_queue.cc:237] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } }
I20240502 13:59:55.541016 27385 raft_consensus.cc:2454] T a734b1d6d928408cac2a9e1b263f2697 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 1.
I20240502 13:59:55.543323 26713 catalog_manager.cc:5484] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68). New cstate: current_term: 1 leader_uuid: "b4fc3c795b1d4505b3a2903532d38b52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: UNKNOWN } } }
I20240502 13:59:55.543757 26713 catalog_manager.cc:5484] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68). New cstate: current_term: 1 leader_uuid: "b4fc3c795b1d4505b3a2903532d38b52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: UNKNOWN } } }
W20240502 13:59:55.544239 27320 consensus_peers.cc:485] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 -> Peer 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71:42201): Couldn't send request to peer 5f9ddfa1f6fc484d8fb96075a8efb649. Status: Network error: Client connection negotiation failed: client connection to 127.24.66.71:42201: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 13:59:55.659428 28051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:55.659684 28051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:55.662909 28051 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240502 13:59:55.749001 26859 raft_consensus.cc:1261] T a734b1d6d928408cac2a9e1b263f2697 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:55.749135 27518 raft_consensus.cc:1261] T a734b1d6d928408cac2a9e1b263f2697 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:55.749361 27519 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:55.749579 27814 consensus_queue.cc:1035] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:55.749720 26989 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:55.749830 27123 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:55.750003 26991 raft_consensus.cc:1261] T df55bedff83e4a5d822547e0fa1c2f87 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:55.750303 26857 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:55.750444 27385 raft_consensus.cc:1261] T a734b1d6d928408cac2a9e1b263f2697 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:55.750470 27123 raft_consensus.cc:1261] T df55bedff83e4a5d822547e0fa1c2f87 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:55.750638 26859 raft_consensus.cc:1261] T df55bedff83e4a5d822547e0fa1c2f87 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:55.750779 27735 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:55.750910 27974 consensus_queue.cc:1035] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:55.751104 27735 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:55.751308 26991 raft_consensus.cc:1261] T a734b1d6d928408cac2a9e1b263f2697 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:55.751385 27735 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:55.751645 27735 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:55.752019 27974 consensus_queue.cc:1035] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:55.759562 27519 raft_consensus.cc:1261] T df55bedff83e4a5d822547e0fa1c2f87 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:55.760026 27814 consensus_queue.cc:1035] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:55.760223 27896 consensus_queue.cc:1035] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:55.760365 27814 consensus_queue.cc:1035] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:55.760658 27896 consensus_queue.cc:1035] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:55.760917 27814 consensus_queue.cc:1035] T df55bedff83e4a5d822547e0fa1c2f87 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 122.367 ms
time per row: 0.0611836 ms
I20240502 13:59:55.840957 26708 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50400:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 13:59:55.848134 26971 tablet_service.cc:1467] Processing CreateTablet for tablet 9efaa3c68c534606bd37bbe4703b2ab2 (DEFAULT_TABLE table=rebalance_test_table_3 [id=ecda844ae2ee4cb5a179b79c8dacb5ac]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.848529 26971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9efaa3c68c534606bd37bbe4703b2ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.849128 27102 tablet_service.cc:1467] Processing CreateTablet for tablet 9efaa3c68c534606bd37bbe4703b2ab2 (DEFAULT_TABLE table=rebalance_test_table_3 [id=ecda844ae2ee4cb5a179b79c8dacb5ac]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.849406 27102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9efaa3c68c534606bd37bbe4703b2ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.850652 28028 tablet_bootstrap.cc:492] T 9efaa3c68c534606bd37bbe4703b2ab2 P 256ce64303ec49e1a5cd8e0ffee436b0: Bootstrap starting.
I20240502 13:59:55.851588 27101 tablet_service.cc:1467] Processing CreateTablet for tablet 6ed88ddf0d1b4dabbbf4604019705436 (DEFAULT_TABLE table=rebalance_test_table_3 [id=ecda844ae2ee4cb5a179b79c8dacb5ac]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.851714 28028 tablet_bootstrap.cc:654] T 9efaa3c68c534606bd37bbe4703b2ab2 P 256ce64303ec49e1a5cd8e0ffee436b0: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.851828 27101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ed88ddf0d1b4dabbbf4604019705436. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.852561 28030 tablet_bootstrap.cc:492] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a: Bootstrap starting.
I20240502 13:59:55.852731 27234 tablet_service.cc:1467] Processing CreateTablet for tablet 9efaa3c68c534606bd37bbe4703b2ab2 (DEFAULT_TABLE table=rebalance_test_table_3 [id=ecda844ae2ee4cb5a179b79c8dacb5ac]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.853027 27234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9efaa3c68c534606bd37bbe4703b2ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.853152 27367 tablet_service.cc:1467] Processing CreateTablet for tablet 9efaa3c68c534606bd37bbe4703b2ab2 (DEFAULT_TABLE table=rebalance_test_table_3 [id=ecda844ae2ee4cb5a179b79c8dacb5ac]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.853363 28030 tablet_bootstrap.cc:654] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.853415 27367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9efaa3c68c534606bd37bbe4703b2ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.854667 27497 tablet_service.cc:1467] Processing CreateTablet for tablet 9efaa3c68c534606bd37bbe4703b2ab2 (DEFAULT_TABLE table=rebalance_test_table_3 [id=ecda844ae2ee4cb5a179b79c8dacb5ac]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.854933 27497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9efaa3c68c534606bd37bbe4703b2ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.855263 27233 tablet_service.cc:1467] Processing CreateTablet for tablet 6ed88ddf0d1b4dabbbf4604019705436 (DEFAULT_TABLE table=rebalance_test_table_3 [id=ecda844ae2ee4cb5a179b79c8dacb5ac]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.855499 27233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ed88ddf0d1b4dabbbf4604019705436. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.855506 28026 tablet_bootstrap.cc:492] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217: Bootstrap starting.
I20240502 13:59:55.856264 28026 tablet_bootstrap.cc:654] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.857162 28025 tablet_bootstrap.cc:492] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136: Bootstrap starting.
I20240502 13:59:55.857446 26971 tablet_service.cc:1467] Processing CreateTablet for tablet 6ed88ddf0d1b4dabbbf4604019705436 (DEFAULT_TABLE table=rebalance_test_table_3 [id=ecda844ae2ee4cb5a179b79c8dacb5ac]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.857566 27366 tablet_service.cc:1467] Processing CreateTablet for tablet 6ed88ddf0d1b4dabbbf4604019705436 (DEFAULT_TABLE table=rebalance_test_table_3 [id=ecda844ae2ee4cb5a179b79c8dacb5ac]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.857704 26971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ed88ddf0d1b4dabbbf4604019705436. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.857797 27366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ed88ddf0d1b4dabbbf4604019705436. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.858063 28025 tablet_bootstrap.cc:654] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.859750 28031 tablet_bootstrap.cc:492] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52: Bootstrap starting.
I20240502 13:59:55.860621 28031 tablet_bootstrap.cc:654] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.860769 28025 tablet_bootstrap.cc:492] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136: No bootstrap required, opened a new log
I20240502 13:59:55.860854 28025 ts_tablet_manager.cc:1367] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 13:59:55.861312 28025 raft_consensus.cc:348] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:55.861430 28025 raft_consensus.cc:374] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.861464 28025 raft_consensus.cc:729] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34b161bd719d4d72baf8da4f40aef136, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.861591 28025 consensus_queue.cc:260] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:55.861668 28031 tablet_bootstrap.cc:492] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52: No bootstrap required, opened a new log
I20240502 13:59:55.861742 28031 ts_tablet_manager.cc:1367] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:55.861912 28025 ts_tablet_manager.cc:1398] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:55.861985 27366 tablet_service.cc:1467] Processing CreateTablet for tablet 1ad8b348f02043dfa4736276dc306895 (DEFAULT_TABLE table=rebalance_test_table_3 [id=ecda844ae2ee4cb5a179b79c8dacb5ac]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.862197 27366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ad8b348f02043dfa4736276dc306895. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.862162 28031 raft_consensus.cc:348] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:55.862265 28031 raft_consensus.cc:374] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.862301 28031 raft_consensus.cc:729] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4fc3c795b1d4505b3a2903532d38b52, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.862426 28031 consensus_queue.cc:260] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:55.862720 28031 ts_tablet_manager.cc:1398] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:55.862885 28031 tablet_bootstrap.cc:492] T 6ed88ddf0d1b4dabbbf4604019705436 P b4fc3c795b1d4505b3a2903532d38b52: Bootstrap starting.
I20240502 13:59:55.863332 27101 tablet_service.cc:1467] Processing CreateTablet for tablet 1ad8b348f02043dfa4736276dc306895 (DEFAULT_TABLE table=rebalance_test_table_3 [id=ecda844ae2ee4cb5a179b79c8dacb5ac]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.863586 27101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ad8b348f02043dfa4736276dc306895. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.864120 26838 tablet_service.cc:1467] Processing CreateTablet for tablet 1ad8b348f02043dfa4736276dc306895 (DEFAULT_TABLE table=rebalance_test_table_3 [id=ecda844ae2ee4cb5a179b79c8dacb5ac]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.864444 26838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ad8b348f02043dfa4736276dc306895. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.864470 27497 tablet_service.cc:1467] Processing CreateTablet for tablet 6ed88ddf0d1b4dabbbf4604019705436 (DEFAULT_TABLE table=rebalance_test_table_3 [id=ecda844ae2ee4cb5a179b79c8dacb5ac]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.864722 27497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ed88ddf0d1b4dabbbf4604019705436. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.864799 27233 tablet_service.cc:1467] Processing CreateTablet for tablet 1ad8b348f02043dfa4736276dc306895 (DEFAULT_TABLE table=rebalance_test_table_3 [id=ecda844ae2ee4cb5a179b79c8dacb5ac]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.865006 27233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ad8b348f02043dfa4736276dc306895. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.866461 28025 tablet_bootstrap.cc:492] T 6ed88ddf0d1b4dabbbf4604019705436 P 34b161bd719d4d72baf8da4f40aef136: Bootstrap starting.
I20240502 13:59:55.866575 28024 tablet_bootstrap.cc:492] T 1ad8b348f02043dfa4736276dc306895 P d3e36186faa442b7bd87357e74d47f68: Bootstrap starting.
I20240502 13:59:55.867542 28025 tablet_bootstrap.cc:654] T 6ed88ddf0d1b4dabbbf4604019705436 P 34b161bd719d4d72baf8da4f40aef136: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.867542 28031 tablet_bootstrap.cc:654] T 6ed88ddf0d1b4dabbbf4604019705436 P b4fc3c795b1d4505b3a2903532d38b52: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.867784 28024 tablet_bootstrap.cc:654] T 1ad8b348f02043dfa4736276dc306895 P d3e36186faa442b7bd87357e74d47f68: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.870046 26971 tablet_service.cc:1467] Processing CreateTablet for tablet 1ad8b348f02043dfa4736276dc306895 (DEFAULT_TABLE table=rebalance_test_table_3 [id=ecda844ae2ee4cb5a179b79c8dacb5ac]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:55.870941 26971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ad8b348f02043dfa4736276dc306895. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:55.874869 28031 tablet_bootstrap.cc:492] T 6ed88ddf0d1b4dabbbf4604019705436 P b4fc3c795b1d4505b3a2903532d38b52: No bootstrap required, opened a new log
I20240502 13:59:55.874965 28031 ts_tablet_manager.cc:1367] T 6ed88ddf0d1b4dabbbf4604019705436 P b4fc3c795b1d4505b3a2903532d38b52: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 13:59:55.876103 28031 raft_consensus.cc:348] T 6ed88ddf0d1b4dabbbf4604019705436 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:55.876235 28031 raft_consensus.cc:374] T 6ed88ddf0d1b4dabbbf4604019705436 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.876260 28031 raft_consensus.cc:729] T 6ed88ddf0d1b4dabbbf4604019705436 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4fc3c795b1d4505b3a2903532d38b52, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.876420 28031 consensus_queue.cc:260] T 6ed88ddf0d1b4dabbbf4604019705436 P b4fc3c795b1d4505b3a2903532d38b52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:55.876760 28031 ts_tablet_manager.cc:1398] T 6ed88ddf0d1b4dabbbf4604019705436 P b4fc3c795b1d4505b3a2903532d38b52: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 13:59:55.876974 28031 tablet_bootstrap.cc:492] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52: Bootstrap starting.
I20240502 13:59:55.879742 28026 tablet_bootstrap.cc:492] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217: No bootstrap required, opened a new log
I20240502 13:59:55.879820 28030 tablet_bootstrap.cc:492] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a: No bootstrap required, opened a new log
I20240502 13:59:55.879863 28026 ts_tablet_manager.cc:1367] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.000s
I20240502 13:59:55.879900 28030 ts_tablet_manager.cc:1367] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.000s
I20240502 13:59:55.880307 28030 raft_consensus.cc:348] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:55.880381 28026 raft_consensus.cc:348] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:55.880477 28028 tablet_bootstrap.cc:492] T 9efaa3c68c534606bd37bbe4703b2ab2 P 256ce64303ec49e1a5cd8e0ffee436b0: No bootstrap required, opened a new log
I20240502 13:59:55.880476 28030 raft_consensus.cc:374] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.880504 28026 raft_consensus.cc:374] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.880519 28030 raft_consensus.cc:729] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0280081e4ff74896832ec3985c372b3a, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.880538 28026 raft_consensus.cc:729] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44f8c16a14a48a38a5a74af2899c217, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.880566 28028 ts_tablet_manager.cc:1367] T 9efaa3c68c534606bd37bbe4703b2ab2 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent bootstrapping tablet: real 0.030s user 0.000s sys 0.002s
I20240502 13:59:55.880637 28030 consensus_queue.cc:260] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:55.880666 28026 consensus_queue.cc:260] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:55.880923 28030 ts_tablet_manager.cc:1398] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 13:59:55.880925 28026 ts_tablet_manager.cc:1398] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:55.880966 28028 raft_consensus.cc:348] T 9efaa3c68c534606bd37bbe4703b2ab2 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:55.881080 28028 raft_consensus.cc:374] T 9efaa3c68c534606bd37bbe4703b2ab2 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.881098 28026 tablet_bootstrap.cc:492] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217: Bootstrap starting.
I20240502 13:59:55.881103 28030 tablet_bootstrap.cc:492] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a: Bootstrap starting.
I20240502 13:59:55.881116 28028 raft_consensus.cc:729] T 9efaa3c68c534606bd37bbe4703b2ab2 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 256ce64303ec49e1a5cd8e0ffee436b0, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.881233 28028 consensus_queue.cc:260] T 9efaa3c68c534606bd37bbe4703b2ab2 P 256ce64303ec49e1a5cd8e0ffee436b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:55.881481 28028 ts_tablet_manager.cc:1398] T 9efaa3c68c534606bd37bbe4703b2ab2 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 13:59:55.881639 28028 tablet_bootstrap.cc:492] T 6ed88ddf0d1b4dabbbf4604019705436 P 256ce64303ec49e1a5cd8e0ffee436b0: Bootstrap starting.
I20240502 13:59:55.881958 28030 tablet_bootstrap.cc:654] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.882481 28028 tablet_bootstrap.cc:654] T 6ed88ddf0d1b4dabbbf4604019705436 P 256ce64303ec49e1a5cd8e0ffee436b0: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.883455 28024 tablet_bootstrap.cc:492] T 1ad8b348f02043dfa4736276dc306895 P d3e36186faa442b7bd87357e74d47f68: No bootstrap required, opened a new log
I20240502 13:59:55.883558 28024 ts_tablet_manager.cc:1367] T 1ad8b348f02043dfa4736276dc306895 P d3e36186faa442b7bd87357e74d47f68: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 13:59:55.883970 28024 raft_consensus.cc:348] T 1ad8b348f02043dfa4736276dc306895 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:55.884083 28024 raft_consensus.cc:374] T 1ad8b348f02043dfa4736276dc306895 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.884114 28024 raft_consensus.cc:729] T 1ad8b348f02043dfa4736276dc306895 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3e36186faa442b7bd87357e74d47f68, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.884236 28024 consensus_queue.cc:260] T 1ad8b348f02043dfa4736276dc306895 P d3e36186faa442b7bd87357e74d47f68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:55.885779 28024 ts_tablet_manager.cc:1398] T 1ad8b348f02043dfa4736276dc306895 P d3e36186faa442b7bd87357e74d47f68: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:55.886588 28026 tablet_bootstrap.cc:654] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.888291 28026 tablet_bootstrap.cc:492] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217: No bootstrap required, opened a new log
I20240502 13:59:55.888444 28026 ts_tablet_manager.cc:1367] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 13:59:55.888885 28026 raft_consensus.cc:348] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:55.889001 28026 raft_consensus.cc:374] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.889088 28026 raft_consensus.cc:729] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44f8c16a14a48a38a5a74af2899c217, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.889258 28026 consensus_queue.cc:260] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:55.889693 28026 ts_tablet_manager.cc:1398] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 13:59:55.889858 28026 tablet_bootstrap.cc:492] T 1ad8b348f02043dfa4736276dc306895 P e44f8c16a14a48a38a5a74af2899c217: Bootstrap starting.
I20240502 13:59:55.890379 28030 tablet_bootstrap.cc:492] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a: No bootstrap required, opened a new log
I20240502 13:59:55.890463 28030 ts_tablet_manager.cc:1367] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20240502 13:59:55.890761 28028 tablet_bootstrap.cc:492] T 6ed88ddf0d1b4dabbbf4604019705436 P 256ce64303ec49e1a5cd8e0ffee436b0: No bootstrap required, opened a new log
I20240502 13:59:55.890839 28026 tablet_bootstrap.cc:654] T 1ad8b348f02043dfa4736276dc306895 P e44f8c16a14a48a38a5a74af2899c217: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.890847 28028 ts_tablet_manager.cc:1367] T 6ed88ddf0d1b4dabbbf4604019705436 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20240502 13:59:55.891312 28028 raft_consensus.cc:348] T 6ed88ddf0d1b4dabbbf4604019705436 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:55.891458 28028 raft_consensus.cc:374] T 6ed88ddf0d1b4dabbbf4604019705436 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.891498 28028 raft_consensus.cc:729] T 6ed88ddf0d1b4dabbbf4604019705436 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 256ce64303ec49e1a5cd8e0ffee436b0, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.891628 28028 consensus_queue.cc:260] T 6ed88ddf0d1b4dabbbf4604019705436 P 256ce64303ec49e1a5cd8e0ffee436b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:55.891824 28026 tablet_bootstrap.cc:492] T 1ad8b348f02043dfa4736276dc306895 P e44f8c16a14a48a38a5a74af2899c217: No bootstrap required, opened a new log
I20240502 13:59:55.891888 28026 ts_tablet_manager.cc:1367] T 1ad8b348f02043dfa4736276dc306895 P e44f8c16a14a48a38a5a74af2899c217: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 13:59:55.891918 28028 ts_tablet_manager.cc:1398] T 6ed88ddf0d1b4dabbbf4604019705436 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 13:59:55.892078 28028 tablet_bootstrap.cc:492] T 1ad8b348f02043dfa4736276dc306895 P 256ce64303ec49e1a5cd8e0ffee436b0: Bootstrap starting.
I20240502 13:59:55.892364 28026 raft_consensus.cc:348] T 1ad8b348f02043dfa4736276dc306895 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:55.892283 28030 raft_consensus.cc:348] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:55.892469 28026 raft_consensus.cc:374] T 1ad8b348f02043dfa4736276dc306895 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.892514 28026 raft_consensus.cc:729] T 1ad8b348f02043dfa4736276dc306895 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44f8c16a14a48a38a5a74af2899c217, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.892522 28030 raft_consensus.cc:374] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.892565 28030 raft_consensus.cc:729] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0280081e4ff74896832ec3985c372b3a, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.892623 28026 consensus_queue.cc:260] T 1ad8b348f02043dfa4736276dc306895 P e44f8c16a14a48a38a5a74af2899c217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:55.892707 28030 consensus_queue.cc:260] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:55.892796 28025 tablet_bootstrap.cc:492] T 6ed88ddf0d1b4dabbbf4604019705436 P 34b161bd719d4d72baf8da4f40aef136: No bootstrap required, opened a new log
I20240502 13:59:55.892877 28025 ts_tablet_manager.cc:1367] T 6ed88ddf0d1b4dabbbf4604019705436 P 34b161bd719d4d72baf8da4f40aef136: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20240502 13:59:55.892911 28026 ts_tablet_manager.cc:1398] T 1ad8b348f02043dfa4736276dc306895 P e44f8c16a14a48a38a5a74af2899c217: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 13:59:55.892992 28030 ts_tablet_manager.cc:1398] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 13:59:55.893292 28025 raft_consensus.cc:348] T 6ed88ddf0d1b4dabbbf4604019705436 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:55.893417 28025 raft_consensus.cc:374] T 6ed88ddf0d1b4dabbbf4604019705436 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.893461 28025 raft_consensus.cc:729] T 6ed88ddf0d1b4dabbbf4604019705436 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34b161bd719d4d72baf8da4f40aef136, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.893592 28025 consensus_queue.cc:260] T 6ed88ddf0d1b4dabbbf4604019705436 P 34b161bd719d4d72baf8da4f40aef136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:55.893899 28025 ts_tablet_manager.cc:1398] T 6ed88ddf0d1b4dabbbf4604019705436 P 34b161bd719d4d72baf8da4f40aef136: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:55.894860 28028 tablet_bootstrap.cc:654] T 1ad8b348f02043dfa4736276dc306895 P 256ce64303ec49e1a5cd8e0ffee436b0: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.896503 28028 tablet_bootstrap.cc:492] T 1ad8b348f02043dfa4736276dc306895 P 256ce64303ec49e1a5cd8e0ffee436b0: No bootstrap required, opened a new log
I20240502 13:59:55.896595 28028 ts_tablet_manager.cc:1367] T 1ad8b348f02043dfa4736276dc306895 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240502 13:59:55.896991 28028 raft_consensus.cc:348] T 1ad8b348f02043dfa4736276dc306895 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:55.897082 28028 raft_consensus.cc:374] T 1ad8b348f02043dfa4736276dc306895 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.897135 28028 raft_consensus.cc:729] T 1ad8b348f02043dfa4736276dc306895 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 256ce64303ec49e1a5cd8e0ffee436b0, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.897187 28031 tablet_bootstrap.cc:654] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.897246 28028 consensus_queue.cc:260] T 1ad8b348f02043dfa4736276dc306895 P 256ce64303ec49e1a5cd8e0ffee436b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:55.897550 28028 ts_tablet_manager.cc:1398] T 1ad8b348f02043dfa4736276dc306895 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:55.897706 28030 tablet_bootstrap.cc:492] T 1ad8b348f02043dfa4736276dc306895 P 0280081e4ff74896832ec3985c372b3a: Bootstrap starting.
I20240502 13:59:55.898846 28031 tablet_bootstrap.cc:492] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52: No bootstrap required, opened a new log
I20240502 13:59:55.898932 28030 tablet_bootstrap.cc:654] T 1ad8b348f02043dfa4736276dc306895 P 0280081e4ff74896832ec3985c372b3a: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:55.899039 28031 ts_tablet_manager.cc:1367] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.001s
I20240502 13:59:55.899533 28031 raft_consensus.cc:348] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:55.899720 28031 raft_consensus.cc:374] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.899827 28031 raft_consensus.cc:729] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4fc3c795b1d4505b3a2903532d38b52, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.900013 28031 consensus_queue.cc:260] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:55.900168 27782 raft_consensus.cc:482] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:55.900265 27782 raft_consensus.cc:504] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:55.900527 28031 ts_tablet_manager.cc:1398] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:55.900903 27782 leader_election.cc:288] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107), 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473), b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68:35167), 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733)
I20240502 13:59:55.900933 28030 tablet_bootstrap.cc:492] T 1ad8b348f02043dfa4736276dc306895 P 0280081e4ff74896832ec3985c372b3a: No bootstrap required, opened a new log
I20240502 13:59:55.901001 28030 ts_tablet_manager.cc:1367] T 1ad8b348f02043dfa4736276dc306895 P 0280081e4ff74896832ec3985c372b3a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 13:59:55.901368 28030 raft_consensus.cc:348] T 1ad8b348f02043dfa4736276dc306895 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:55.901469 28030 raft_consensus.cc:374] T 1ad8b348f02043dfa4736276dc306895 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:55.901508 28030 raft_consensus.cc:729] T 1ad8b348f02043dfa4736276dc306895 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0280081e4ff74896832ec3985c372b3a, State: Initialized, Role: FOLLOWER
I20240502 13:59:55.901621 28030 consensus_queue.cc:260] T 1ad8b348f02043dfa4736276dc306895 P 0280081e4ff74896832ec3985c372b3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:55.901779 27123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9efaa3c68c534606bd37bbe4703b2ab2" candidate_uuid: "34b161bd719d4d72baf8da4f40aef136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0280081e4ff74896832ec3985c372b3a" is_pre_election: true
I20240502 13:59:55.901866 28030 ts_tablet_manager.cc:1398] T 1ad8b348f02043dfa4736276dc306895 P 0280081e4ff74896832ec3985c372b3a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 13:59:55.901844 27385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9efaa3c68c534606bd37bbe4703b2ab2" candidate_uuid: "34b161bd719d4d72baf8da4f40aef136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44f8c16a14a48a38a5a74af2899c217" is_pre_election: true
I20240502 13:59:55.901892 27123 raft_consensus.cc:2454] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34b161bd719d4d72baf8da4f40aef136 in term 0.
I20240502 13:59:55.901963 27385 raft_consensus.cc:2454] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34b161bd719d4d72baf8da4f40aef136 in term 0.
I20240502 13:59:55.902159 26989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9efaa3c68c534606bd37bbe4703b2ab2" candidate_uuid: "34b161bd719d4d72baf8da4f40aef136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" is_pre_election: true
I20240502 13:59:55.902266 26989 raft_consensus.cc:2454] T 9efaa3c68c534606bd37bbe4703b2ab2 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34b161bd719d4d72baf8da4f40aef136 in term 0.
I20240502 13:59:55.902398 27453 leader_election.cc:302] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0280081e4ff74896832ec3985c372b3a, 34b161bd719d4d72baf8da4f40aef136, e44f8c16a14a48a38a5a74af2899c217; no voters:
I20240502 13:59:55.902611 27782 raft_consensus.cc:2781] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:55.902662 27782 raft_consensus.cc:482] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:55.902702 27782 raft_consensus.cc:3037] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:55.906989 27255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9efaa3c68c534606bd37bbe4703b2ab2" candidate_uuid: "34b161bd719d4d72baf8da4f40aef136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4fc3c795b1d4505b3a2903532d38b52" is_pre_election: true
I20240502 13:59:55.907127 27255 raft_consensus.cc:2454] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34b161bd719d4d72baf8da4f40aef136 in term 0.
I20240502 13:59:55.916613 27989 raft_consensus.cc:482] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:55.916735 27989 raft_consensus.cc:504] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:55.917441 27989 leader_election.cc:288] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107), 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473), b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68:35167), 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217)
I20240502 13:59:55.917667 27385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ed88ddf0d1b4dabbbf4604019705436" candidate_uuid: "0280081e4ff74896832ec3985c372b3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44f8c16a14a48a38a5a74af2899c217" is_pre_election: true
I20240502 13:59:55.917845 27385 raft_consensus.cc:2454] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0280081e4ff74896832ec3985c372b3a in term 0.
I20240502 13:59:55.918387 27255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ed88ddf0d1b4dabbbf4604019705436" candidate_uuid: "0280081e4ff74896832ec3985c372b3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4fc3c795b1d4505b3a2903532d38b52" is_pre_election: true
I20240502 13:59:55.918442 27517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ed88ddf0d1b4dabbbf4604019705436" candidate_uuid: "0280081e4ff74896832ec3985c372b3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b161bd719d4d72baf8da4f40aef136" is_pre_election: true
I20240502 13:59:55.918521 27255 raft_consensus.cc:2454] T 6ed88ddf0d1b4dabbbf4604019705436 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0280081e4ff74896832ec3985c372b3a in term 0.
I20240502 13:59:55.918570 27517 raft_consensus.cc:2454] T 6ed88ddf0d1b4dabbbf4604019705436 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0280081e4ff74896832ec3985c372b3a in term 0.
I20240502 13:59:55.918861 27056 leader_election.cc:302] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0280081e4ff74896832ec3985c372b3a, 34b161bd719d4d72baf8da4f40aef136, e44f8c16a14a48a38a5a74af2899c217; no voters:
I20240502 13:59:55.919023 27989 raft_consensus.cc:2781] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:55.919098 27989 raft_consensus.cc:482] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:55.919147 27989 raft_consensus.cc:3037] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:55.922045 26989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ed88ddf0d1b4dabbbf4604019705436" candidate_uuid: "0280081e4ff74896832ec3985c372b3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" is_pre_election: true
I20240502 13:59:55.922169 26989 raft_consensus.cc:2454] T 6ed88ddf0d1b4dabbbf4604019705436 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0280081e4ff74896832ec3985c372b3a in term 0.
I20240502 13:59:55.937662 27782 raft_consensus.cc:504] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:55.937647 27989 raft_consensus.cc:504] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:55.938234 27782 leader_election.cc:288] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [CANDIDATE]: Term 1 election: Requested vote from peers e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107), 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473), b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68:35167), 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733)
I20240502 13:59:55.938346 27989 leader_election.cc:288] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [CANDIDATE]: Term 1 election: Requested vote from peers e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107), 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473), b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68:35167), 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217)
I20240502 13:59:55.939042 27517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ed88ddf0d1b4dabbbf4604019705436" candidate_uuid: "0280081e4ff74896832ec3985c372b3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b161bd719d4d72baf8da4f40aef136"
I20240502 13:59:55.939177 27517 raft_consensus.cc:3037] T 6ed88ddf0d1b4dabbbf4604019705436 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:55.939616 27255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9efaa3c68c534606bd37bbe4703b2ab2" candidate_uuid: "34b161bd719d4d72baf8da4f40aef136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4fc3c795b1d4505b3a2903532d38b52"
I20240502 13:59:55.939736 27255 raft_consensus.cc:3037] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:55.940537 27517 raft_consensus.cc:2454] T 6ed88ddf0d1b4dabbbf4604019705436 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0280081e4ff74896832ec3985c372b3a in term 1.
I20240502 13:59:55.940804 26991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9efaa3c68c534606bd37bbe4703b2ab2" candidate_uuid: "34b161bd719d4d72baf8da4f40aef136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0"
I20240502 13:59:55.940935 26991 raft_consensus.cc:3037] T 9efaa3c68c534606bd37bbe4703b2ab2 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:55.941277 27255 raft_consensus.cc:2454] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34b161bd719d4d72baf8da4f40aef136 in term 1.
I20240502 13:59:55.941895 27385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9efaa3c68c534606bd37bbe4703b2ab2" candidate_uuid: "34b161bd719d4d72baf8da4f40aef136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44f8c16a14a48a38a5a74af2899c217"
I20240502 13:59:55.942018 27385 raft_consensus.cc:3037] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:55.942421 26989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ed88ddf0d1b4dabbbf4604019705436" candidate_uuid: "0280081e4ff74896832ec3985c372b3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0"
I20240502 13:59:55.942548 26989 raft_consensus.cc:3037] T 6ed88ddf0d1b4dabbbf4604019705436 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:55.942822 27123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9efaa3c68c534606bd37bbe4703b2ab2" candidate_uuid: "34b161bd719d4d72baf8da4f40aef136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0280081e4ff74896832ec3985c372b3a"
I20240502 13:59:55.942942 27123 raft_consensus.cc:3037] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:55.943163 27387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ed88ddf0d1b4dabbbf4604019705436" candidate_uuid: "0280081e4ff74896832ec3985c372b3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44f8c16a14a48a38a5a74af2899c217"
I20240502 13:59:55.943265 27387 raft_consensus.cc:3037] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:55.943301 27255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ed88ddf0d1b4dabbbf4604019705436" candidate_uuid: "0280081e4ff74896832ec3985c372b3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4fc3c795b1d4505b3a2903532d38b52"
I20240502 13:59:55.943434 27255 raft_consensus.cc:3037] T 6ed88ddf0d1b4dabbbf4604019705436 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:55.969673 27896 raft_consensus.cc:482] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:55.969808 27896 raft_consensus.cc:504] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:55.970284 27896 leader_election.cc:288] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841), 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473), 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733), e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107)
I20240502 13:59:55.970678 27386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ad8b348f02043dfa4736276dc306895" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44f8c16a14a48a38a5a74af2899c217" is_pre_election: true
I20240502 13:59:55.970695 27122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ad8b348f02043dfa4736276dc306895" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0280081e4ff74896832ec3985c372b3a" is_pre_election: true
I20240502 13:59:55.970736 26859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ad8b348f02043dfa4736276dc306895" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e36186faa442b7bd87357e74d47f68" is_pre_election: true
I20240502 13:59:55.970855 26859 raft_consensus.cc:2454] T 1ad8b348f02043dfa4736276dc306895 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 0.
I20240502 13:59:55.970854 27122 raft_consensus.cc:2454] T 1ad8b348f02043dfa4736276dc306895 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 0.
I20240502 13:59:55.970851 27386 raft_consensus.cc:2454] T 1ad8b348f02043dfa4736276dc306895 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 0.
I20240502 13:59:55.970861 26990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ad8b348f02043dfa4736276dc306895" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" is_pre_election: true
I20240502 13:59:55.970988 26990 raft_consensus.cc:2454] T 1ad8b348f02043dfa4736276dc306895 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 0.
I20240502 13:59:55.971206 27189 leader_election.cc:302] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: b4fc3c795b1d4505b3a2903532d38b52, d3e36186faa442b7bd87357e74d47f68, e44f8c16a14a48a38a5a74af2899c217; no voters:
I20240502 13:59:55.971402 27896 raft_consensus.cc:2781] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:55.971482 27896 raft_consensus.cc:482] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:55.971525 27896 raft_consensus.cc:3037] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Advancing to term 1
W20240502 13:59:56.010987 27320 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.66.71:42201: connect: Connection refused (error 111) [suppressed 10 similar messages]
I20240502 13:59:56.035022 27123 raft_consensus.cc:2454] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34b161bd719d4d72baf8da4f40aef136 in term 1.
I20240502 13:59:56.035022 27385 raft_consensus.cc:2454] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34b161bd719d4d72baf8da4f40aef136 in term 1.
I20240502 13:59:56.035022 26991 raft_consensus.cc:2454] T 9efaa3c68c534606bd37bbe4703b2ab2 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34b161bd719d4d72baf8da4f40aef136 in term 1.
I20240502 13:59:56.035513 27455 leader_election.cc:302] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0280081e4ff74896832ec3985c372b3a, 34b161bd719d4d72baf8da4f40aef136, b4fc3c795b1d4505b3a2903532d38b52; no voters:
I20240502 13:59:56.035985 27255 raft_consensus.cc:2454] T 6ed88ddf0d1b4dabbbf4604019705436 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0280081e4ff74896832ec3985c372b3a in term 1.
I20240502 13:59:56.036197 27387 raft_consensus.cc:2454] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0280081e4ff74896832ec3985c372b3a in term 1.
I20240502 13:59:56.036386 27782 raft_consensus.cc:2781] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:56.036572 27782 raft_consensus.cc:686] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [term 1 LEADER]: Becoming Leader. State: Replica: 34b161bd719d4d72baf8da4f40aef136, State: Running, Role: LEADER
I20240502 13:59:56.036592 27059 leader_election.cc:302] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0280081e4ff74896832ec3985c372b3a, 34b161bd719d4d72baf8da4f40aef136, b4fc3c795b1d4505b3a2903532d38b52; no voters:
I20240502 13:59:56.036757 27782 consensus_queue.cc:237] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:56.036890 26989 raft_consensus.cc:2454] T 6ed88ddf0d1b4dabbbf4604019705436 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0280081e4ff74896832ec3985c372b3a in term 1.
I20240502 13:59:56.037187 27989 raft_consensus.cc:2781] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:56.037341 27989 raft_consensus.cc:686] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [term 1 LEADER]: Becoming Leader. State: Replica: 0280081e4ff74896832ec3985c372b3a, State: Running, Role: LEADER
I20240502 13:59:56.037510 27989 consensus_queue.cc:237] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } }
I20240502 13:59:56.037751 27896 raft_consensus.cc:504] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:56.038283 27896 leader_election.cc:288] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [CANDIDATE]: Term 1 election: Requested vote from peers d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841), 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473), 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733), e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107)
I20240502 13:59:56.038661 27387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ad8b348f02043dfa4736276dc306895" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e44f8c16a14a48a38a5a74af2899c217"
I20240502 13:59:56.038789 27387 raft_consensus.cc:3037] T 1ad8b348f02043dfa4736276dc306895 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:56.039052 27123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ad8b348f02043dfa4736276dc306895" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0280081e4ff74896832ec3985c372b3a"
I20240502 13:59:56.039167 27123 raft_consensus.cc:3037] T 1ad8b348f02043dfa4736276dc306895 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:56.039355 26859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ad8b348f02043dfa4736276dc306895" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e36186faa442b7bd87357e74d47f68"
I20240502 13:59:56.039470 26859 raft_consensus.cc:3037] T 1ad8b348f02043dfa4736276dc306895 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:56.039969 26711 catalog_manager.cc:5484] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70). New cstate: current_term: 1 leader_uuid: "34b161bd719d4d72baf8da4f40aef136" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: HEALTHY } } }
I20240502 13:59:56.040643 26989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ad8b348f02043dfa4736276dc306895" candidate_uuid: "b4fc3c795b1d4505b3a2903532d38b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0"
I20240502 13:59:56.040766 26989 raft_consensus.cc:3037] T 1ad8b348f02043dfa4736276dc306895 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:56.041859 26712 catalog_manager.cc:5484] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0280081e4ff74896832ec3985c372b3a (127.24.66.67). New cstate: current_term: 1 leader_uuid: "0280081e4ff74896832ec3985c372b3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: UNKNOWN } } }
I20240502 13:59:56.129873 26859 raft_consensus.cc:2454] T 1ad8b348f02043dfa4736276dc306895 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 1.
I20240502 13:59:56.129869 27387 raft_consensus.cc:2454] T 1ad8b348f02043dfa4736276dc306895 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 1.
I20240502 13:59:56.129861 27123 raft_consensus.cc:2454] T 1ad8b348f02043dfa4736276dc306895 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 1.
I20240502 13:59:56.130260 27191 leader_election.cc:302] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0280081e4ff74896832ec3985c372b3a, b4fc3c795b1d4505b3a2903532d38b52, e44f8c16a14a48a38a5a74af2899c217; no voters:
I20240502 13:59:56.130437 26989 raft_consensus.cc:2454] T 1ad8b348f02043dfa4736276dc306895 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4fc3c795b1d4505b3a2903532d38b52 in term 1.
I20240502 13:59:56.130493 27896 raft_consensus.cc:2781] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:56.130661 27896 raft_consensus.cc:686] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 LEADER]: Becoming Leader. State: Replica: b4fc3c795b1d4505b3a2903532d38b52, State: Running, Role: LEADER
I20240502 13:59:56.130825 27896 consensus_queue.cc:237] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:56.132879 26712 catalog_manager.cc:5484] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68). New cstate: current_term: 1 leader_uuid: "b4fc3c795b1d4505b3a2903532d38b52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: UNKNOWN } } }
W20240502 13:59:56.287434 28132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:56.287683 28132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:56.290731 28132 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20240502 13:59:56.372973 27387 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:56.373595 27517 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:56.373662 27123 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:56.373886 27255 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:56.374478 27938 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:56.374882 27782 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:56.375155 26989 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:56.375211 27938 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:56.375351 27255 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:56.375662 27387 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:56.375780 27938 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:56.377025 28128 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:56.377341 28128 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:56.380957 27387 raft_consensus.cc:1261] T 1ad8b348f02043dfa4736276dc306895 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:56.381311 26990 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:56.381615 27782 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:56.381727 27122 raft_consensus.cc:1261] T 1ad8b348f02043dfa4736276dc306895 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:56.385006 26859 raft_consensus.cc:1261] T 1ad8b348f02043dfa4736276dc306895 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:56.385411 27989 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:56.385847 26991 raft_consensus.cc:1261] T 1ad8b348f02043dfa4736276dc306895 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:56.386667 28048 consensus_queue.cc:1035] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:56.386948 28048 consensus_queue.cc:1035] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:56.387348 28048 consensus_queue.cc:1035] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:56.391846 27896 consensus_queue.cc:1035] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 173.235 ms
time per row: 0.0866175 ms
I20240502 13:59:56.508654 26714 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50400:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 5
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 13:59:56.514609 26971 tablet_service.cc:1467] Processing CreateTablet for tablet fcf8a440cfbe490e9aa735cad599e472 (DEFAULT_TABLE table=rebalance_test_table_4 [id=56abfd3d24fd434b996a35f2174a0641]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:56.514673 26838 tablet_service.cc:1467] Processing CreateTablet for tablet fcf8a440cfbe490e9aa735cad599e472 (DEFAULT_TABLE table=rebalance_test_table_4 [id=56abfd3d24fd434b996a35f2174a0641]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:56.514961 26971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcf8a440cfbe490e9aa735cad599e472. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:56.514994 26838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcf8a440cfbe490e9aa735cad599e472. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:56.515081 27497 tablet_service.cc:1467] Processing CreateTablet for tablet fcf8a440cfbe490e9aa735cad599e472 (DEFAULT_TABLE table=rebalance_test_table_4 [id=56abfd3d24fd434b996a35f2174a0641]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:56.515337 27497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcf8a440cfbe490e9aa735cad599e472. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:56.516705 27101 tablet_service.cc:1467] Processing CreateTablet for tablet fcf8a440cfbe490e9aa735cad599e472 (DEFAULT_TABLE table=rebalance_test_table_4 [id=56abfd3d24fd434b996a35f2174a0641]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:56.516808 26837 tablet_service.cc:1467] Processing CreateTablet for tablet f7ea57a2ffe04e9e962c61d0e82fc475 (DEFAULT_TABLE table=rebalance_test_table_4 [id=56abfd3d24fd434b996a35f2174a0641]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:56.516978 27101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcf8a440cfbe490e9aa735cad599e472. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:56.517010 26837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7ea57a2ffe04e9e962c61d0e82fc475. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:56.517256 27366 tablet_service.cc:1467] Processing CreateTablet for tablet f7ea57a2ffe04e9e962c61d0e82fc475 (DEFAULT_TABLE table=rebalance_test_table_4 [id=56abfd3d24fd434b996a35f2174a0641]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:56.517542 27366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7ea57a2ffe04e9e962c61d0e82fc475. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:56.517701 27367 tablet_service.cc:1467] Processing CreateTablet for tablet d6fcbe92870d40a6b4a860f5f4f81c29 (DEFAULT_TABLE table=rebalance_test_table_4 [id=56abfd3d24fd434b996a35f2174a0641]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:56.517903 27367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6fcbe92870d40a6b4a860f5f4f81c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:56.518332 28206 tablet_bootstrap.cc:492] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136: Bootstrap starting.
I20240502 13:59:56.519248 28206 tablet_bootstrap.cc:654] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:56.519459 26837 tablet_service.cc:1467] Processing CreateTablet for tablet d6fcbe92870d40a6b4a860f5f4f81c29 (DEFAULT_TABLE table=rebalance_test_table_4 [id=56abfd3d24fd434b996a35f2174a0641]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:56.519536 26970 tablet_service.cc:1467] Processing CreateTablet for tablet f7ea57a2ffe04e9e962c61d0e82fc475 (DEFAULT_TABLE table=rebalance_test_table_4 [id=56abfd3d24fd434b996a35f2174a0641]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:56.519778 26970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7ea57a2ffe04e9e962c61d0e82fc475. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:56.519779 26837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6fcbe92870d40a6b4a860f5f4f81c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:56.519835 27497 tablet_service.cc:1467] Processing CreateTablet for tablet f7ea57a2ffe04e9e962c61d0e82fc475 (DEFAULT_TABLE table=rebalance_test_table_4 [id=56abfd3d24fd434b996a35f2174a0641]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:56.520051 27497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7ea57a2ffe04e9e962c61d0e82fc475. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:56.519114 28207 tablet_bootstrap.cc:492] T fcf8a440cfbe490e9aa735cad599e472 P d3e36186faa442b7bd87357e74d47f68: Bootstrap starting.
I20240502 13:59:56.520633 26969 tablet_service.cc:1467] Processing CreateTablet for tablet d6fcbe92870d40a6b4a860f5f4f81c29 (DEFAULT_TABLE table=rebalance_test_table_4 [id=56abfd3d24fd434b996a35f2174a0641]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:56.520876 26969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6fcbe92870d40a6b4a860f5f4f81c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:56.521597 28207 tablet_bootstrap.cc:654] T fcf8a440cfbe490e9aa735cad599e472 P d3e36186faa442b7bd87357e74d47f68: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:56.522642 27101 tablet_service.cc:1467] Processing CreateTablet for tablet f7ea57a2ffe04e9e962c61d0e82fc475 (DEFAULT_TABLE table=rebalance_test_table_4 [id=56abfd3d24fd434b996a35f2174a0641]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:56.522648 27233 tablet_service.cc:1467] Processing CreateTablet for tablet fcf8a440cfbe490e9aa735cad599e472 (DEFAULT_TABLE table=rebalance_test_table_4 [id=56abfd3d24fd434b996a35f2174a0641]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:56.522735 28207 tablet_bootstrap.cc:492] T fcf8a440cfbe490e9aa735cad599e472 P d3e36186faa442b7bd87357e74d47f68: No bootstrap required, opened a new log
I20240502 13:59:56.522806 28207 ts_tablet_manager.cc:1367] T fcf8a440cfbe490e9aa735cad599e472 P d3e36186faa442b7bd87357e74d47f68: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 13:59:56.522902 27101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7ea57a2ffe04e9e962c61d0e82fc475. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:56.522931 27233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcf8a440cfbe490e9aa735cad599e472. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:56.523036 27499 tablet_service.cc:1467] Processing CreateTablet for tablet d6fcbe92870d40a6b4a860f5f4f81c29 (DEFAULT_TABLE table=rebalance_test_table_4 [id=56abfd3d24fd434b996a35f2174a0641]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:56.523250 27499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6fcbe92870d40a6b4a860f5f4f81c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:56.523253 28207 raft_consensus.cc:348] T fcf8a440cfbe490e9aa735cad599e472 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:56.523355 28207 raft_consensus.cc:374] T fcf8a440cfbe490e9aa735cad599e472 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:56.523381 28207 raft_consensus.cc:729] T fcf8a440cfbe490e9aa735cad599e472 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3e36186faa442b7bd87357e74d47f68, State: Initialized, Role: FOLLOWER
I20240502 13:59:56.523511 28207 consensus_queue.cc:260] T fcf8a440cfbe490e9aa735cad599e472 P d3e36186faa442b7bd87357e74d47f68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:56.523784 28207 ts_tablet_manager.cc:1398] T fcf8a440cfbe490e9aa735cad599e472 P d3e36186faa442b7bd87357e74d47f68: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:56.523953 28207 tablet_bootstrap.cc:492] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68: Bootstrap starting.
I20240502 13:59:56.524730 28207 tablet_bootstrap.cc:654] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:56.525130 28212 tablet_bootstrap.cc:492] T fcf8a440cfbe490e9aa735cad599e472 P b4fc3c795b1d4505b3a2903532d38b52: Bootstrap starting.
I20240502 13:59:56.525341 27234 tablet_service.cc:1467] Processing CreateTablet for tablet d6fcbe92870d40a6b4a860f5f4f81c29 (DEFAULT_TABLE table=rebalance_test_table_4 [id=56abfd3d24fd434b996a35f2174a0641]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 13:59:56.525563 27234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6fcbe92870d40a6b4a860f5f4f81c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 13:59:56.525799 28211 tablet_bootstrap.cc:492] T d6fcbe92870d40a6b4a860f5f4f81c29 P 256ce64303ec49e1a5cd8e0ffee436b0: Bootstrap starting.
I20240502 13:59:56.526012 28212 tablet_bootstrap.cc:654] T fcf8a440cfbe490e9aa735cad599e472 P b4fc3c795b1d4505b3a2903532d38b52: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:56.526842 28211 tablet_bootstrap.cc:654] T d6fcbe92870d40a6b4a860f5f4f81c29 P 256ce64303ec49e1a5cd8e0ffee436b0: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:56.528558 28208 tablet_bootstrap.cc:492] T fcf8a440cfbe490e9aa735cad599e472 P 0280081e4ff74896832ec3985c372b3a: Bootstrap starting.
I20240502 13:59:56.528564 28212 tablet_bootstrap.cc:492] T fcf8a440cfbe490e9aa735cad599e472 P b4fc3c795b1d4505b3a2903532d38b52: No bootstrap required, opened a new log
I20240502 13:59:56.528654 28212 ts_tablet_manager.cc:1367] T fcf8a440cfbe490e9aa735cad599e472 P b4fc3c795b1d4505b3a2903532d38b52: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 13:59:56.529098 28212 raft_consensus.cc:348] T fcf8a440cfbe490e9aa735cad599e472 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:56.529224 28212 raft_consensus.cc:374] T fcf8a440cfbe490e9aa735cad599e472 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:56.529270 28212 raft_consensus.cc:729] T fcf8a440cfbe490e9aa735cad599e472 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4fc3c795b1d4505b3a2903532d38b52, State: Initialized, Role: FOLLOWER
I20240502 13:59:56.529397 28212 consensus_queue.cc:260] T fcf8a440cfbe490e9aa735cad599e472 P b4fc3c795b1d4505b3a2903532d38b52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:56.529551 28208 tablet_bootstrap.cc:654] T fcf8a440cfbe490e9aa735cad599e472 P 0280081e4ff74896832ec3985c372b3a: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:56.529186 28211 tablet_bootstrap.cc:492] T d6fcbe92870d40a6b4a860f5f4f81c29 P 256ce64303ec49e1a5cd8e0ffee436b0: No bootstrap required, opened a new log
I20240502 13:59:56.529704 28212 ts_tablet_manager.cc:1398] T fcf8a440cfbe490e9aa735cad599e472 P b4fc3c795b1d4505b3a2903532d38b52: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:56.529841 28211 ts_tablet_manager.cc:1367] T d6fcbe92870d40a6b4a860f5f4f81c29 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 13:59:56.529883 28212 tablet_bootstrap.cc:492] T d6fcbe92870d40a6b4a860f5f4f81c29 P b4fc3c795b1d4505b3a2903532d38b52: Bootstrap starting.
I20240502 13:59:56.530629 28206 tablet_bootstrap.cc:492] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136: No bootstrap required, opened a new log
I20240502 13:59:56.530726 28206 ts_tablet_manager.cc:1367] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 13:59:56.530725 28212 tablet_bootstrap.cc:654] T d6fcbe92870d40a6b4a860f5f4f81c29 P b4fc3c795b1d4505b3a2903532d38b52: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:56.531251 28206 raft_consensus.cc:348] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:56.531356 28206 raft_consensus.cc:374] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:56.531386 28206 raft_consensus.cc:729] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34b161bd719d4d72baf8da4f40aef136, State: Initialized, Role: FOLLOWER
I20240502 13:59:56.531512 28206 consensus_queue.cc:260] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:56.531759 28206 ts_tablet_manager.cc:1398] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:56.531937 28206 tablet_bootstrap.cc:492] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136: Bootstrap starting.
I20240502 13:59:56.532318 28212 tablet_bootstrap.cc:492] T d6fcbe92870d40a6b4a860f5f4f81c29 P b4fc3c795b1d4505b3a2903532d38b52: No bootstrap required, opened a new log
I20240502 13:59:56.532418 28212 ts_tablet_manager.cc:1367] T d6fcbe92870d40a6b4a860f5f4f81c29 P b4fc3c795b1d4505b3a2903532d38b52: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 13:59:56.532840 28206 tablet_bootstrap.cc:654] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:56.532837 28212 raft_consensus.cc:348] T d6fcbe92870d40a6b4a860f5f4f81c29 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.532938 28212 raft_consensus.cc:374] T d6fcbe92870d40a6b4a860f5f4f81c29 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:56.532975 28212 raft_consensus.cc:729] T d6fcbe92870d40a6b4a860f5f4f81c29 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4fc3c795b1d4505b3a2903532d38b52, State: Initialized, Role: FOLLOWER
I20240502 13:59:56.533097 28212 consensus_queue.cc:260] T d6fcbe92870d40a6b4a860f5f4f81c29 P b4fc3c795b1d4505b3a2903532d38b52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.533409 28212 ts_tablet_manager.cc:1398] T d6fcbe92870d40a6b4a860f5f4f81c29 P b4fc3c795b1d4505b3a2903532d38b52: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:56.533823 28211 raft_consensus.cc:348] T d6fcbe92870d40a6b4a860f5f4f81c29 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.533950 28211 raft_consensus.cc:374] T d6fcbe92870d40a6b4a860f5f4f81c29 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:56.533986 28211 raft_consensus.cc:729] T d6fcbe92870d40a6b4a860f5f4f81c29 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 256ce64303ec49e1a5cd8e0ffee436b0, State: Initialized, Role: FOLLOWER
I20240502 13:59:56.534112 28211 consensus_queue.cc:260] T d6fcbe92870d40a6b4a860f5f4f81c29 P 256ce64303ec49e1a5cd8e0ffee436b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.534394 28211 ts_tablet_manager.cc:1398] T d6fcbe92870d40a6b4a860f5f4f81c29 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent starting tablet: real 0.004s user 0.000s sys 0.000s
I20240502 13:59:56.534600 28211 tablet_bootstrap.cc:492] T f7ea57a2ffe04e9e962c61d0e82fc475 P 256ce64303ec49e1a5cd8e0ffee436b0: Bootstrap starting.
I20240502 13:59:56.535025 28208 tablet_bootstrap.cc:492] T fcf8a440cfbe490e9aa735cad599e472 P 0280081e4ff74896832ec3985c372b3a: No bootstrap required, opened a new log
I20240502 13:59:56.535108 28208 ts_tablet_manager.cc:1367] T fcf8a440cfbe490e9aa735cad599e472 P 0280081e4ff74896832ec3985c372b3a: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 13:59:56.535521 28211 tablet_bootstrap.cc:654] T f7ea57a2ffe04e9e962c61d0e82fc475 P 256ce64303ec49e1a5cd8e0ffee436b0: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:56.535568 28208 raft_consensus.cc:348] T fcf8a440cfbe490e9aa735cad599e472 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:56.535688 28208 raft_consensus.cc:374] T fcf8a440cfbe490e9aa735cad599e472 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:56.535724 28208 raft_consensus.cc:729] T fcf8a440cfbe490e9aa735cad599e472 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0280081e4ff74896832ec3985c372b3a, State: Initialized, Role: FOLLOWER
I20240502 13:59:56.535849 28208 consensus_queue.cc:260] T fcf8a440cfbe490e9aa735cad599e472 P 0280081e4ff74896832ec3985c372b3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:56.536149 28208 ts_tablet_manager.cc:1398] T fcf8a440cfbe490e9aa735cad599e472 P 0280081e4ff74896832ec3985c372b3a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:56.536461 28208 tablet_bootstrap.cc:492] T f7ea57a2ffe04e9e962c61d0e82fc475 P 0280081e4ff74896832ec3985c372b3a: Bootstrap starting.
I20240502 13:59:56.537638 28208 tablet_bootstrap.cc:654] T f7ea57a2ffe04e9e962c61d0e82fc475 P 0280081e4ff74896832ec3985c372b3a: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:56.539111 28208 tablet_bootstrap.cc:492] T f7ea57a2ffe04e9e962c61d0e82fc475 P 0280081e4ff74896832ec3985c372b3a: No bootstrap required, opened a new log
I20240502 13:59:56.539201 28208 ts_tablet_manager.cc:1367] T f7ea57a2ffe04e9e962c61d0e82fc475 P 0280081e4ff74896832ec3985c372b3a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 13:59:56.539757 28208 raft_consensus.cc:348] T f7ea57a2ffe04e9e962c61d0e82fc475 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.539906 28208 raft_consensus.cc:374] T f7ea57a2ffe04e9e962c61d0e82fc475 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:56.539947 28208 raft_consensus.cc:729] T f7ea57a2ffe04e9e962c61d0e82fc475 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0280081e4ff74896832ec3985c372b3a, State: Initialized, Role: FOLLOWER
I20240502 13:59:56.540124 28208 consensus_queue.cc:260] T f7ea57a2ffe04e9e962c61d0e82fc475 P 0280081e4ff74896832ec3985c372b3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.540632 28208 ts_tablet_manager.cc:1398] T f7ea57a2ffe04e9e962c61d0e82fc475 P 0280081e4ff74896832ec3985c372b3a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 13:59:56.541215 28211 tablet_bootstrap.cc:492] T f7ea57a2ffe04e9e962c61d0e82fc475 P 256ce64303ec49e1a5cd8e0ffee436b0: No bootstrap required, opened a new log
I20240502 13:59:56.541316 28211 ts_tablet_manager.cc:1367] T f7ea57a2ffe04e9e962c61d0e82fc475 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 13:59:56.541859 28211 raft_consensus.cc:348] T f7ea57a2ffe04e9e962c61d0e82fc475 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.541971 28211 raft_consensus.cc:374] T f7ea57a2ffe04e9e962c61d0e82fc475 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:56.542013 28211 raft_consensus.cc:729] T f7ea57a2ffe04e9e962c61d0e82fc475 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 256ce64303ec49e1a5cd8e0ffee436b0, State: Initialized, Role: FOLLOWER
I20240502 13:59:56.542146 28211 consensus_queue.cc:260] T f7ea57a2ffe04e9e962c61d0e82fc475 P 256ce64303ec49e1a5cd8e0ffee436b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.544453 28211 ts_tablet_manager.cc:1398] T f7ea57a2ffe04e9e962c61d0e82fc475 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 13:59:56.544680 28211 tablet_bootstrap.cc:492] T fcf8a440cfbe490e9aa735cad599e472 P 256ce64303ec49e1a5cd8e0ffee436b0: Bootstrap starting.
I20240502 13:59:56.545603 28211 tablet_bootstrap.cc:654] T fcf8a440cfbe490e9aa735cad599e472 P 256ce64303ec49e1a5cd8e0ffee436b0: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:56.546335 28221 tablet_bootstrap.cc:492] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217: Bootstrap starting.
I20240502 13:59:56.546780 28211 tablet_bootstrap.cc:492] T fcf8a440cfbe490e9aa735cad599e472 P 256ce64303ec49e1a5cd8e0ffee436b0: No bootstrap required, opened a new log
I20240502 13:59:56.546860 28211 ts_tablet_manager.cc:1367] T fcf8a440cfbe490e9aa735cad599e472 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:56.547207 28221 tablet_bootstrap.cc:654] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:56.547447 28211 raft_consensus.cc:348] T fcf8a440cfbe490e9aa735cad599e472 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:56.547572 28211 raft_consensus.cc:374] T fcf8a440cfbe490e9aa735cad599e472 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:56.547616 28211 raft_consensus.cc:729] T fcf8a440cfbe490e9aa735cad599e472 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 256ce64303ec49e1a5cd8e0ffee436b0, State: Initialized, Role: FOLLOWER
I20240502 13:59:56.547756 28211 consensus_queue.cc:260] T fcf8a440cfbe490e9aa735cad599e472 P 256ce64303ec49e1a5cd8e0ffee436b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:56.547998 28211 ts_tablet_manager.cc:1398] T fcf8a440cfbe490e9aa735cad599e472 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:56.548137 28221 tablet_bootstrap.cc:492] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217: No bootstrap required, opened a new log
I20240502 13:59:56.548231 28221 ts_tablet_manager.cc:1367] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:56.548712 28221 raft_consensus.cc:348] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.548830 28221 raft_consensus.cc:374] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:56.548871 28221 raft_consensus.cc:729] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44f8c16a14a48a38a5a74af2899c217, State: Initialized, Role: FOLLOWER
I20240502 13:59:56.548867 28207 tablet_bootstrap.cc:492] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68: No bootstrap required, opened a new log
I20240502 13:59:56.548985 28207 ts_tablet_manager.cc:1367] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.000s
I20240502 13:59:56.549003 28221 consensus_queue.cc:260] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.549268 28221 ts_tablet_manager.cc:1398] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 13:59:56.549440 28221 tablet_bootstrap.cc:492] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217: Bootstrap starting.
I20240502 13:59:56.549434 28207 raft_consensus.cc:348] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.549542 28207 raft_consensus.cc:374] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:56.549573 28207 raft_consensus.cc:729] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3e36186faa442b7bd87357e74d47f68, State: Initialized, Role: FOLLOWER
I20240502 13:59:56.549686 28207 consensus_queue.cc:260] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.549986 28207 ts_tablet_manager.cc:1398] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:56.550146 28207 tablet_bootstrap.cc:492] T d6fcbe92870d40a6b4a860f5f4f81c29 P d3e36186faa442b7bd87357e74d47f68: Bootstrap starting.
I20240502 13:59:56.550321 28221 tablet_bootstrap.cc:654] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:56.551062 28207 tablet_bootstrap.cc:654] T d6fcbe92870d40a6b4a860f5f4f81c29 P d3e36186faa442b7bd87357e74d47f68: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:56.552177 28221 tablet_bootstrap.cc:492] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217: No bootstrap required, opened a new log
I20240502 13:59:56.552263 28221 ts_tablet_manager.cc:1367] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 13:59:56.552605 28207 tablet_bootstrap.cc:492] T d6fcbe92870d40a6b4a860f5f4f81c29 P d3e36186faa442b7bd87357e74d47f68: No bootstrap required, opened a new log
I20240502 13:59:56.552691 28207 ts_tablet_manager.cc:1367] T d6fcbe92870d40a6b4a860f5f4f81c29 P d3e36186faa442b7bd87357e74d47f68: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 13:59:56.553120 28207 raft_consensus.cc:348] T d6fcbe92870d40a6b4a860f5f4f81c29 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.553242 28207 raft_consensus.cc:374] T d6fcbe92870d40a6b4a860f5f4f81c29 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:56.553289 28207 raft_consensus.cc:729] T d6fcbe92870d40a6b4a860f5f4f81c29 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3e36186faa442b7bd87357e74d47f68, State: Initialized, Role: FOLLOWER
I20240502 13:59:56.553447 28207 consensus_queue.cc:260] T d6fcbe92870d40a6b4a860f5f4f81c29 P d3e36186faa442b7bd87357e74d47f68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.553768 28207 ts_tablet_manager.cc:1398] T d6fcbe92870d40a6b4a860f5f4f81c29 P d3e36186faa442b7bd87357e74d47f68: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 13:59:56.553860 28221 raft_consensus.cc:348] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.553966 28221 raft_consensus.cc:374] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:56.553992 28221 raft_consensus.cc:729] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44f8c16a14a48a38a5a74af2899c217, State: Initialized, Role: FOLLOWER
I20240502 13:59:56.554116 28221 consensus_queue.cc:260] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.554359 28221 ts_tablet_manager.cc:1398] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 13:59:56.560729 28206 tablet_bootstrap.cc:492] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136: No bootstrap required, opened a new log
I20240502 13:59:56.560829 28206 ts_tablet_manager.cc:1367] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20240502 13:59:56.561484 28206 raft_consensus.cc:348] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.561630 28206 raft_consensus.cc:374] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:56.561684 28206 raft_consensus.cc:729] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34b161bd719d4d72baf8da4f40aef136, State: Initialized, Role: FOLLOWER
I20240502 13:59:56.561837 28206 consensus_queue.cc:260] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.562150 28206 ts_tablet_manager.cc:1398] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:56.562399 28206 tablet_bootstrap.cc:492] T d6fcbe92870d40a6b4a860f5f4f81c29 P 34b161bd719d4d72baf8da4f40aef136: Bootstrap starting.
I20240502 13:59:56.563339 28206 tablet_bootstrap.cc:654] T d6fcbe92870d40a6b4a860f5f4f81c29 P 34b161bd719d4d72baf8da4f40aef136: Neither blocks nor log segments found. Creating new log.
I20240502 13:59:56.564487 28206 tablet_bootstrap.cc:492] T d6fcbe92870d40a6b4a860f5f4f81c29 P 34b161bd719d4d72baf8da4f40aef136: No bootstrap required, opened a new log
I20240502 13:59:56.564580 28206 ts_tablet_manager.cc:1367] T d6fcbe92870d40a6b4a860f5f4f81c29 P 34b161bd719d4d72baf8da4f40aef136: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:56.565023 28206 raft_consensus.cc:348] T d6fcbe92870d40a6b4a860f5f4f81c29 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.565140 28206 raft_consensus.cc:374] T d6fcbe92870d40a6b4a860f5f4f81c29 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 13:59:56.565181 28206 raft_consensus.cc:729] T d6fcbe92870d40a6b4a860f5f4f81c29 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34b161bd719d4d72baf8da4f40aef136, State: Initialized, Role: FOLLOWER
I20240502 13:59:56.565315 28206 consensus_queue.cc:260] T d6fcbe92870d40a6b4a860f5f4f81c29 P 34b161bd719d4d72baf8da4f40aef136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.565595 28206 ts_tablet_manager.cc:1398] T d6fcbe92870d40a6b4a860f5f4f81c29 P 34b161bd719d4d72baf8da4f40aef136: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 13:59:56.572396 28105 raft_consensus.cc:482] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:56.572517 28105 raft_consensus.cc:504] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.572973 28105 leader_election.cc:288] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217), b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68:35167), d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841), 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473)
I20240502 13:59:56.573231 27517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fcbe92870d40a6b4a860f5f4f81c29" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b161bd719d4d72baf8da4f40aef136" is_pre_election: true
I20240502 13:59:56.573277 27255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fcbe92870d40a6b4a860f5f4f81c29" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4fc3c795b1d4505b3a2903532d38b52" is_pre_election: true
I20240502 13:59:56.573360 27517 raft_consensus.cc:2454] T d6fcbe92870d40a6b4a860f5f4f81c29 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 0.
I20240502 13:59:56.573392 26989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fcbe92870d40a6b4a860f5f4f81c29" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" is_pre_election: true
I20240502 13:59:56.573432 27255 raft_consensus.cc:2454] T d6fcbe92870d40a6b4a860f5f4f81c29 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 0.
I20240502 13:59:56.573527 26989 raft_consensus.cc:2454] T d6fcbe92870d40a6b4a860f5f4f81c29 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 0.
I20240502 13:59:56.573733 27323 leader_election.cc:302] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 34b161bd719d4d72baf8da4f40aef136, b4fc3c795b1d4505b3a2903532d38b52, e44f8c16a14a48a38a5a74af2899c217; no voters:
I20240502 13:59:56.573777 26859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fcbe92870d40a6b4a860f5f4f81c29" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e36186faa442b7bd87357e74d47f68" is_pre_election: true
I20240502 13:59:56.573913 28105 raft_consensus.cc:2781] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:56.573915 26859 raft_consensus.cc:2454] T d6fcbe92870d40a6b4a860f5f4f81c29 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 0.
I20240502 13:59:56.573963 28105 raft_consensus.cc:482] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:56.573990 28105 raft_consensus.cc:3037] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:56.587278 28198 raft_consensus.cc:482] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:56.587388 28198 raft_consensus.cc:504] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:56.587853 28198 leader_election.cc:288] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473), d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841), 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733), b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68:35167)
I20240502 13:59:56.588167 26859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf8a440cfbe490e9aa735cad599e472" candidate_uuid: "34b161bd719d4d72baf8da4f40aef136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e36186faa442b7bd87357e74d47f68" is_pre_election: true
I20240502 13:59:56.588202 27122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf8a440cfbe490e9aa735cad599e472" candidate_uuid: "34b161bd719d4d72baf8da4f40aef136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0280081e4ff74896832ec3985c372b3a" is_pre_election: true
I20240502 13:59:56.588188 26989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf8a440cfbe490e9aa735cad599e472" candidate_uuid: "34b161bd719d4d72baf8da4f40aef136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" is_pre_election: true
I20240502 13:59:56.588275 27255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf8a440cfbe490e9aa735cad599e472" candidate_uuid: "34b161bd719d4d72baf8da4f40aef136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4fc3c795b1d4505b3a2903532d38b52" is_pre_election: true
I20240502 13:59:56.588305 26859 raft_consensus.cc:2454] T fcf8a440cfbe490e9aa735cad599e472 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34b161bd719d4d72baf8da4f40aef136 in term 0.
I20240502 13:59:56.588320 27122 raft_consensus.cc:2454] T fcf8a440cfbe490e9aa735cad599e472 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34b161bd719d4d72baf8da4f40aef136 in term 0.
I20240502 13:59:56.588411 27255 raft_consensus.cc:2454] T fcf8a440cfbe490e9aa735cad599e472 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34b161bd719d4d72baf8da4f40aef136 in term 0.
I20240502 13:59:56.588320 26989 raft_consensus.cc:2454] T fcf8a440cfbe490e9aa735cad599e472 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34b161bd719d4d72baf8da4f40aef136 in term 0.
I20240502 13:59:56.588721 27455 leader_election.cc:302] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 0280081e4ff74896832ec3985c372b3a, 34b161bd719d4d72baf8da4f40aef136, b4fc3c795b1d4505b3a2903532d38b52; no voters:
I20240502 13:59:56.588886 28198 raft_consensus.cc:2781] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:56.588958 28198 raft_consensus.cc:482] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:56.588997 28198 raft_consensus.cc:3037] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:56.591936 28227 raft_consensus.cc:482] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 13:59:56.592027 28227 raft_consensus.cc:504] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.592497 28227 leader_election.cc:288] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841), 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217), 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733), 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473)
I20240502 13:59:56.592770 27517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7ea57a2ffe04e9e962c61d0e82fc475" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b161bd719d4d72baf8da4f40aef136" is_pre_election: true
I20240502 13:59:56.592805 26989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7ea57a2ffe04e9e962c61d0e82fc475" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" is_pre_election: true
I20240502 13:59:56.592900 27517 raft_consensus.cc:2454] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 0.
I20240502 13:59:56.592926 26989 raft_consensus.cc:2454] T f7ea57a2ffe04e9e962c61d0e82fc475 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 0.
I20240502 13:59:56.592919 27122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7ea57a2ffe04e9e962c61d0e82fc475" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0280081e4ff74896832ec3985c372b3a" is_pre_election: true
I20240502 13:59:56.592936 26859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7ea57a2ffe04e9e962c61d0e82fc475" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e36186faa442b7bd87357e74d47f68" is_pre_election: true
I20240502 13:59:56.593017 27122 raft_consensus.cc:2454] T f7ea57a2ffe04e9e962c61d0e82fc475 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 0.
I20240502 13:59:56.593039 26859 raft_consensus.cc:2454] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 0.
I20240502 13:59:56.593250 27320 leader_election.cc:302] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 256ce64303ec49e1a5cd8e0ffee436b0, 34b161bd719d4d72baf8da4f40aef136, e44f8c16a14a48a38a5a74af2899c217; no voters:
I20240502 13:59:56.593417 28227 raft_consensus.cc:2781] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 13:59:56.593478 28227 raft_consensus.cc:482] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 13:59:56.593519 28227 raft_consensus.cc:3037] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:56.638159 28198 raft_consensus.cc:504] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:56.638157 28105 raft_consensus.cc:504] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.638710 28198 leader_election.cc:288] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [CANDIDATE]: Term 1 election: Requested vote from peers 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473), d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841), 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733), b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68:35167)
I20240502 13:59:56.638767 28105 leader_election.cc:288] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [CANDIDATE]: Term 1 election: Requested vote from peers 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217), b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68:35167), d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841), 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473)
I20240502 13:59:56.638947 28227 raft_consensus.cc:504] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.639292 26859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf8a440cfbe490e9aa735cad599e472" candidate_uuid: "34b161bd719d4d72baf8da4f40aef136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e36186faa442b7bd87357e74d47f68"
I20240502 13:59:56.639384 28227 leader_election.cc:288] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [CANDIDATE]: Term 1 election: Requested vote from peers d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841), 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217), 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733), 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473)
I20240502 13:59:56.639420 26859 raft_consensus.cc:3037] T fcf8a440cfbe490e9aa735cad599e472 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:56.639950 27519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7ea57a2ffe04e9e962c61d0e82fc475" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b161bd719d4d72baf8da4f40aef136"
I20240502 13:59:56.639991 26989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf8a440cfbe490e9aa735cad599e472" candidate_uuid: "34b161bd719d4d72baf8da4f40aef136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0"
I20240502 13:59:56.640056 27519 raft_consensus.cc:3037] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:56.640098 26989 raft_consensus.cc:3037] T fcf8a440cfbe490e9aa735cad599e472 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:56.640626 26859 raft_consensus.cc:2454] T fcf8a440cfbe490e9aa735cad599e472 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34b161bd719d4d72baf8da4f40aef136 in term 1.
I20240502 13:59:56.640901 26991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fcbe92870d40a6b4a860f5f4f81c29" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0"
I20240502 13:59:56.641016 27519 raft_consensus.cc:2454] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 1.
I20240502 13:59:56.641016 26991 raft_consensus.cc:3037] T d6fcbe92870d40a6b4a860f5f4f81c29 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:56.641357 27517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fcbe92870d40a6b4a860f5f4f81c29" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b161bd719d4d72baf8da4f40aef136"
I20240502 13:59:56.641461 27517 raft_consensus.cc:3037] T d6fcbe92870d40a6b4a860f5f4f81c29 P 34b161bd719d4d72baf8da4f40aef136 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:56.642109 26991 raft_consensus.cc:2454] T d6fcbe92870d40a6b4a860f5f4f81c29 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 1.
I20240502 13:59:56.642341 26990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7ea57a2ffe04e9e962c61d0e82fc475" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "256ce64303ec49e1a5cd8e0ffee436b0"
I20240502 13:59:56.642436 26990 raft_consensus.cc:3037] T f7ea57a2ffe04e9e962c61d0e82fc475 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:56.641023 26989 raft_consensus.cc:2454] T fcf8a440cfbe490e9aa735cad599e472 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34b161bd719d4d72baf8da4f40aef136 in term 1.
I20240502 13:59:56.642696 27517 raft_consensus.cc:2454] T d6fcbe92870d40a6b4a860f5f4f81c29 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 1.
I20240502 13:59:56.643013 27320 leader_election.cc:302] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 256ce64303ec49e1a5cd8e0ffee436b0, 34b161bd719d4d72baf8da4f40aef136, e44f8c16a14a48a38a5a74af2899c217; no voters:
I20240502 13:59:56.643059 27122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf8a440cfbe490e9aa735cad599e472" candidate_uuid: "34b161bd719d4d72baf8da4f40aef136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0280081e4ff74896832ec3985c372b3a"
I20240502 13:59:56.643168 28227 raft_consensus.cc:2781] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:56.643169 27122 raft_consensus.cc:3037] T fcf8a440cfbe490e9aa735cad599e472 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:56.643239 28227 raft_consensus.cc:686] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [term 1 LEADER]: Becoming Leader. State: Replica: e44f8c16a14a48a38a5a74af2899c217, State: Running, Role: LEADER
I20240502 13:59:56.643375 28227 consensus_queue.cc:237] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.643544 26990 raft_consensus.cc:2454] T f7ea57a2ffe04e9e962c61d0e82fc475 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 1.
I20240502 13:59:56.643922 27255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcf8a440cfbe490e9aa735cad599e472" candidate_uuid: "34b161bd719d4d72baf8da4f40aef136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4fc3c795b1d4505b3a2903532d38b52"
I20240502 13:59:56.644038 27255 raft_consensus.cc:3037] T fcf8a440cfbe490e9aa735cad599e472 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:56.644145 27452 leader_election.cc:302] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 256ce64303ec49e1a5cd8e0ffee436b0, 34b161bd719d4d72baf8da4f40aef136, d3e36186faa442b7bd87357e74d47f68; no voters:
I20240502 13:59:56.644170 27320 leader_election.cc:302] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 5 voters: 3 yes votes; 0 no votes. yes voters: 256ce64303ec49e1a5cd8e0ffee436b0, 34b161bd719d4d72baf8da4f40aef136, e44f8c16a14a48a38a5a74af2899c217; no voters:
I20240502 13:59:56.644299 28227 raft_consensus.cc:2781] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:56.644349 28198 raft_consensus.cc:2781] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 13:59:56.644388 28227 raft_consensus.cc:686] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [term 1 LEADER]: Becoming Leader. State: Replica: e44f8c16a14a48a38a5a74af2899c217, State: Running, Role: LEADER
I20240502 13:59:56.644418 28198 raft_consensus.cc:686] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [term 1 LEADER]: Becoming Leader. State: Replica: 34b161bd719d4d72baf8da4f40aef136, State: Running, Role: LEADER
I20240502 13:59:56.644515 28227 consensus_queue.cc:237] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } }
I20240502 13:59:56.644539 28198 consensus_queue.cc:237] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } }
I20240502 13:59:56.644649 26858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7ea57a2ffe04e9e962c61d0e82fc475" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e36186faa442b7bd87357e74d47f68"
I20240502 13:59:56.644759 26858 raft_consensus.cc:3037] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:56.645948 26859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fcbe92870d40a6b4a860f5f4f81c29" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3e36186faa442b7bd87357e74d47f68"
I20240502 13:59:56.646046 26859 raft_consensus.cc:3037] T d6fcbe92870d40a6b4a860f5f4f81c29 P d3e36186faa442b7bd87357e74d47f68 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:56.646322 26713 catalog_manager.cc:5484] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 reported cstate change: term changed from 0 to 1, leader changed from <none> to e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69). New cstate: current_term: 1 leader_uuid: "e44f8c16a14a48a38a5a74af2899c217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: UNKNOWN } } }
I20240502 13:59:56.646623 26714 catalog_manager.cc:5484] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70). New cstate: current_term: 1 leader_uuid: "34b161bd719d4d72baf8da4f40aef136" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: UNKNOWN } } }
I20240502 13:59:56.646696 27254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6fcbe92870d40a6b4a860f5f4f81c29" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4fc3c795b1d4505b3a2903532d38b52"
I20240502 13:59:56.646842 27254 raft_consensus.cc:3037] T d6fcbe92870d40a6b4a860f5f4f81c29 P b4fc3c795b1d4505b3a2903532d38b52 [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:56.646822 27123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7ea57a2ffe04e9e962c61d0e82fc475" candidate_uuid: "e44f8c16a14a48a38a5a74af2899c217" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0280081e4ff74896832ec3985c372b3a"
I20240502 13:59:56.646929 27123 raft_consensus.cc:3037] T f7ea57a2ffe04e9e962c61d0e82fc475 P 0280081e4ff74896832ec3985c372b3a [term 0 FOLLOWER]: Advancing to term 1
I20240502 13:59:56.649711 26714 catalog_manager.cc:5484] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 reported cstate change: term changed from 0 to 1, leader changed from <none> to e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69). New cstate: current_term: 1 leader_uuid: "e44f8c16a14a48a38a5a74af2899c217" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: UNKNOWN } } }
I20240502 13:59:56.735534 27122 raft_consensus.cc:2454] T fcf8a440cfbe490e9aa735cad599e472 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34b161bd719d4d72baf8da4f40aef136 in term 1.
I20240502 13:59:56.735523 27255 raft_consensus.cc:2454] T fcf8a440cfbe490e9aa735cad599e472 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34b161bd719d4d72baf8da4f40aef136 in term 1.
I20240502 13:59:56.736658 27123 raft_consensus.cc:2454] T f7ea57a2ffe04e9e962c61d0e82fc475 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 1.
I20240502 13:59:56.736877 27254 raft_consensus.cc:2454] T d6fcbe92870d40a6b4a860f5f4f81c29 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 1.
I20240502 13:59:56.737152 26858 raft_consensus.cc:2454] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 1.
I20240502 13:59:56.738015 26859 raft_consensus.cc:2454] T d6fcbe92870d40a6b4a860f5f4f81c29 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e44f8c16a14a48a38a5a74af2899c217 in term 1.
W20240502 13:59:56.764592 28235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:56.764814 28235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:56.767828 28235 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20240502 13:59:56.863346 26990 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:56.863472 27123 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:56.863955 28227 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:56.864204 28105 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:56.868528 27517 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:56.868667 27254 raft_consensus.cc:1261] T d6fcbe92870d40a6b4a860f5f4f81c29 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:56.868845 26859 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:56.869163 28228 consensus_queue.cc:1035] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:56.870483 28228 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:56.870664 26990 raft_consensus.cc:1261] T d6fcbe92870d40a6b4a860f5f4f81c29 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:56.870786 28228 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:56.871274 27517 raft_consensus.cc:1261] T d6fcbe92870d40a6b4a860f5f4f81c29 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:56.871343 26859 raft_consensus.cc:1261] T d6fcbe92870d40a6b4a860f5f4f81c29 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:56.872866 28227 consensus_queue.cc:1035] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:56.873162 28227 consensus_queue.cc:1035] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:56.873275 28228 consensus_queue.cc:1035] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:56.884402 28187 mvcc.cc:205] Tried to move back new op lower bound from 7023240793560887296 to 7023240792653160448. Current Snapshot: MvccSnapshot[applied={T|T < 7023240793560887296}]
I20240502 13:59:56.900463 28184 mvcc.cc:205] Tried to move back new op lower bound from 7023240793548742656 to 7023240792657874944. Current Snapshot: MvccSnapshot[applied={T|T < 7023240793548742656}]
I20240502 13:59:56.901403 27122 raft_consensus.cc:1261] T fcf8a440cfbe490e9aa735cad599e472 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:56.901844 28198 consensus_queue.cc:1035] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:56.908069 27254 raft_consensus.cc:1261] T fcf8a440cfbe490e9aa735cad599e472 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:56.909346 28198 consensus_queue.cc:1035] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:56.918190 26989 raft_consensus.cc:1261] T fcf8a440cfbe490e9aa735cad599e472 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:56.918244 26858 raft_consensus.cc:1261] T fcf8a440cfbe490e9aa735cad599e472 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 13:59:56.918746 28198 consensus_queue.cc:1035] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 13:59:56.919067 28128 consensus_queue.cc:1035] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 186.072 ms
time per row: 0.0930361 ms
I20240502 13:59:56.996142 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.71:42201
--local_ip_for_outbound_sockets=127.24.66.71
--tserver_master_addrs=127.24.66.126:45215
--webserver_port=46589
--webserver_interface=127.24.66.71
--builtin_ntp_servers=127.24.66.84:37497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 13:59:57.110961 28279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 13:59:57.111234 28279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 13:59:57.111292 28279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 13:59:57.115545 28279 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 13:59:57.115633 28279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 13:59:57.115726 28279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.71
I20240502 13:59:57.120811 28279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:37497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.71:42201
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.24.66.71
--webserver_port=46589
--tserver_master_addrs=127.24.66.126:45215
--never_fsync=true
--heap_profile_path=/tmp/kudu.28279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.71
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 13:59:57.121896 28279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 13:59:57.123206 28279 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 13:59:57.132300 28284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:57.132913 28287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 13:59:57.133843 28285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 13:59:57.134716 28279 server_base.cc:1034] running on GCE node
I20240502 13:59:57.135076 28279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 13:59:57.135696 28279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 13:59:57.136866 28279 hybrid_clock.cc:648] HybridClock initialized: now 1714658397136847 us; error 30 us; skew 500 ppm
I20240502 13:59:57.138607 28279 webserver.cc:460] Webserver started at http://127.24.66.71:46589/ using document root <none> and password file <none>
I20240502 13:59:57.139108 28279 fs_manager.cc:359] Metadata directory not provided
I20240502 13:59:57.139173 28279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 13:59:57.142417 28279 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 13:59:57.144222 28295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 13:59:57.145197 28279 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 13:59:57.145285 28279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "5f9ddfa1f6fc484d8fb96075a8efb649"
format_stamp: "Formatted at 2024-05-02 13:59:49 on dist-test-slave-5npg"
I20240502 13:59:57.145704 28279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 13:59:57.166350 28279 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 13:59:57.166887 28279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 13:59:57.167006 28279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 13:59:57.167438 28279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 13:59:57.168421 28302 ts_tablet_manager.cc:516] Loading tablet metadata (0/1 complete)
I20240502 13:59:57.171458 28279 ts_tablet_manager.cc:559] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20240502 13:59:57.171515 28279 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20240502 13:59:57.171576 28279 ts_tablet_manager.cc:574] Registering tablets (0/1 complete)
I20240502 13:59:57.172953 28279 ts_tablet_manager.cc:590] Registered 1 tablets
I20240502 13:59:57.172996 28279 ts_tablet_manager.cc:569] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20240502 13:59:57.173229 28302 tablet_bootstrap.cc:492] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap starting.
I20240502 13:59:57.182632 28279 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.71:42201
I20240502 13:59:57.182679 28410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.71:42201 every 8 connection(s)
I20240502 13:59:57.183624 28279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 13:59:57.187678 28302 log.cc:824] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649: Log is configured to *not* fsync() on all Append() calls
I20240502 13:59:57.188877 28302 tablet_bootstrap.cc:492] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 13:59:57.189131 28302 tablet_bootstrap.cc:492] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap complete.
I20240502 13:59:57.190155 28302 ts_tablet_manager.cc:1367] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.012s
I20240502 13:59:57.191684 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 28279
I20240502 13:59:57.191854 24841 test_util.cc:268] Using random seed: 78446253
I20240502 13:59:57.192072 28411 heartbeater.cc:347] Connected to a master server at 127.24.66.126:45215
I20240502 13:59:57.192268 28411 heartbeater.cc:464] Registering TS with master...
I20240502 13:59:57.192991 28411 heartbeater.cc:511] Master 127.24.66.126:45215 requested a full tablet report, sending...
I20240502 13:59:57.192960 28302 raft_consensus.cc:348] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:57.193372 28302 raft_consensus.cc:729] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9ddfa1f6fc484d8fb96075a8efb649, State: Initialized, Role: FOLLOWER
I20240502 13:59:57.193819 28302 consensus_queue.cc:260] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } }
I20240502 13:59:57.194617 28302 ts_tablet_manager.cc:1398] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649: Time spent starting tablet: real 0.004s user 0.004s sys 0.002s
I20240502 13:59:57.195572 26714 ts_manager.cc:194] Re-registered known tserver with Master: 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71:42201)
I20240502 13:59:57.196521 26714 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.71:49321
I20240502 13:59:57.198256 28411 heartbeater.cc:503] Master 127.24.66.126:45215 was elected leader, sending a full tablet report...
I20240502 13:59:57.202827 24841 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_0 already exists
I20240502 13:59:57.203001 24841 test_util.cc:268] Using random seed: 78457400
I20240502 13:59:57.212277 24841 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_1 already exists
I20240502 13:59:57.212452 24841 test_util.cc:268] Using random seed: 78466850
I20240502 13:59:57.230387 24841 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_2 already exists
I20240502 13:59:57.230567 24841 test_util.cc:268] Using random seed: 78484964
I20240502 13:59:57.264530 24841 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_3 already exists
I20240502 13:59:57.264703 24841 test_util.cc:268] Using random seed: 78519100
I20240502 13:59:57.342458 24841 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_4 already exists
W20240502 13:59:57.435475 28412 tablet.cc:2369] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 13:59:58.581352 27101 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:58.616714 26971 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:58.620568 27499 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 13:59:58.690164 26837 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:58.711284 27234 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 13:59:58.736652 28345 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20240502 13:59:58.741874 27188 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.66.66:40473, user_credentials={real_user=slave}} blocked reactor thread for 67929.7us
I20240502 13:59:58.790361 27367 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 13:59:59.543396 26971 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:59.573249 27499 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 13:59:59.606611 28345 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 13:59:59.645109 27367 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 13:59:59.682550 26837 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 13:59:59.714114 27101 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 13:59:59.748937 27234 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:00:00.280496 27519 consensus_queue.cc:237] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 55, Committed index: 55, Last appended: 1.55, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:00.283354 27123 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 55. Preceding OpId from leader: term: 1 index: 56. (index mismatch)
I20240502 14:00:00.284411 28615 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 56, Last known committed idx: 54, Time since last communication: 0.000s
I20240502 14:00:00.284801 27386 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 55. Preceding OpId from leader: term: 1 index: 56. (index mismatch)
I20240502 14:00:00.285305 28615 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 56, Last known committed idx: 55, Time since last communication: 0.000s
I20240502 14:00:00.287600 28615 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [term 1 LEADER]: Committing config change with OpId 1.56: config changed from index -1 to 56, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.288223 27252 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 55. Preceding OpId from leader: term: 1 index: 56. (index mismatch)
I20240502 14:00:00.288717 28614 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 56, Last known committed idx: 54, Time since last communication: 0.000s
I20240502 14:00:00.288187 27120 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.56: config changed from index -1 to 56, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.290372 27386 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.56: config changed from index -1 to 56, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.300941 26714 catalog_manager.cc:5484] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217 reported cstate change: config changed from index -1 to 56, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New cstate: current_term: 1 leader_uuid: "34b161bd719d4d72baf8da4f40aef136" committed_config { opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.319144 27253 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.56: config changed from index -1 to 56, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.329947 27252 consensus_queue.cc:237] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 44, Committed index: 44, Last appended: 1.44, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:00.352488 27383 raft_consensus.cc:1261] T 95f064e56ee241dba95f9a6cd5984d97 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 44. Preceding OpId from leader: term: 1 index: 45. (index mismatch)
I20240502 14:00:00.353878 27119 raft_consensus.cc:1261] T 95f064e56ee241dba95f9a6cd5984d97 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 44. Preceding OpId from leader: term: 1 index: 45. (index mismatch)
I20240502 14:00:00.354758 28500 consensus_queue.cc:1035] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 45, Last known committed idx: 44, Time since last communication: 0.000s
I20240502 14:00:00.357482 28048 consensus_queue.cc:1035] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 45, Last known committed idx: 44, Time since last communication: 0.000s
I20240502 14:00:00.360594 28617 raft_consensus.cc:2932] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 LEADER]: Committing config change with OpId 1.45: config changed from index -1 to 45, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.363636 27383 raft_consensus.cc:2932] T 95f064e56ee241dba95f9a6cd5984d97 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.45: config changed from index -1 to 45, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.364303 26712 catalog_manager.cc:5484] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 reported cstate change: config changed from index -1 to 45, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New cstate: current_term: 1 leader_uuid: "b4fc3c795b1d4505b3a2903532d38b52" committed_config { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:00.367262 27120 raft_consensus.cc:2932] T 95f064e56ee241dba95f9a6cd5984d97 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.45: config changed from index -1 to 45, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
W20240502 14:00:00.373920 27452 consensus_peers.cc:485] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 -> Peer 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71:42201): Couldn't send request to peer 5f9ddfa1f6fc484d8fb96075a8efb649. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9efaa3c68c534606bd37bbe4703b2ab2. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:00.392812 27122 consensus_queue.cc:237] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 49, Committed index: 49, Last appended: 1.49, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 50 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: true } }
I20240502 14:00:00.395054 26859 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 49. Preceding OpId from leader: term: 1 index: 50. (index mismatch)
I20240502 14:00:00.395478 28547 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 50, Last known committed idx: 49, Time since last communication: 0.000s
I20240502 14:00:00.395578 27518 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 49. Preceding OpId from leader: term: 1 index: 50. (index mismatch)
I20240502 14:00:00.395982 28547 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 50, Last known committed idx: 49, Time since last communication: 0.000s
I20240502 14:00:00.398119 27385 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 49. Preceding OpId from leader: term: 1 index: 50. (index mismatch)
I20240502 14:00:00.398875 28547 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [term 1 LEADER]: Committing config change with OpId 1.50: config changed from index -1 to 50, NON_VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) added. New config: { opid_index: 50 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: true } } }
I20240502 14:00:00.399511 26859 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.50: config changed from index -1 to 50, NON_VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) added. New config: { opid_index: 50 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: true } } }
I20240502 14:00:00.401326 28493 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 50, Last known committed idx: 49, Time since last communication: 0.000s
I20240502 14:00:00.402169 26712 catalog_manager.cc:5484] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a reported cstate change: config changed from index -1 to 50, NON_VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) added. New cstate: current_term: 1 leader_uuid: "0280081e4ff74896832ec3985c372b3a" committed_config { opid_index: 50 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:00.402724 27519 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.50: config changed from index -1 to 50, NON_VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) added. New config: { opid_index: 50 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: true } } }
I20240502 14:00:00.407131 27385 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.50: config changed from index -1 to 50, NON_VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) added. New config: { opid_index: 50 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: true } } }
I20240502 14:00:00.426519 27519 raft_consensus.cc:1261] T 95f064e56ee241dba95f9a6cd5984d97 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 44. Preceding OpId from leader: term: 1 index: 45. (index mismatch)
I20240502 14:00:00.426992 26858 raft_consensus.cc:1261] T 95f064e56ee241dba95f9a6cd5984d97 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 44. Preceding OpId from leader: term: 1 index: 45. (index mismatch)
I20240502 14:00:00.445318 28617 consensus_queue.cc:1035] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 45, Last known committed idx: 44, Time since last communication: 0.000s
I20240502 14:00:00.455211 28531 consensus_queue.cc:1035] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 45, Last known committed idx: 44, Time since last communication: 0.000s
I20240502 14:00:00.456951 26859 raft_consensus.cc:2932] T 95f064e56ee241dba95f9a6cd5984d97 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.45: config changed from index -1 to 45, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.463359 27252 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 49. Preceding OpId from leader: term: 1 index: 50. (index mismatch)
I20240502 14:00:00.472787 28493 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 50, Last known committed idx: 49, Time since last communication: 0.000s
I20240502 14:00:00.474449 27517 raft_consensus.cc:2932] T 95f064e56ee241dba95f9a6cd5984d97 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.45: config changed from index -1 to 45, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.513053 26985 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 53. Preceding OpId from leader: term: 1 index: 56. (index mismatch)
I20240502 14:00:00.526861 28615 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 56, Last known committed idx: 52, Time since last communication: 0.000s
I20240502 14:00:00.534336 27252 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.50: config changed from index -1 to 50, NON_VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) added. New config: { opid_index: 50 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: true } } }
W20240502 14:00:00.551462 27056 consensus_peers.cc:485] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a -> Peer 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473): Couldn't send request to peer 256ce64303ec49e1a5cd8e0ffee436b0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4a74c0c312a7403d89958a3632cab7d3. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:00.568120 26985 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.56: config changed from index -1 to 56, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.626821 27120 consensus_queue.cc:237] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 65, Committed index: 65, Last appended: 1.65, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:00.650177 27386 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
I20240502 14:00:00.650655 28493 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 65, Time since last communication: 0.000s
I20240502 14:00:00.661521 26987 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 64. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
W20240502 14:00:00.665767 27056 consensus_peers.cc:485] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a -> Peer 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71:42201): Couldn't send request to peer 5f9ddfa1f6fc484d8fb96075a8efb649. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6ed88ddf0d1b4dabbbf4604019705436. This is attempt 1: this message will repeat every 5th retry.
W20240502 14:00:00.668751 27188 consensus_peers.cc:485] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 -> Peer 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71:42201): Couldn't send request to peer 5f9ddfa1f6fc484d8fb96075a8efb649. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 95f064e56ee241dba95f9a6cd5984d97. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:00.682267 27518 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
I20240502 14:00:00.682355 27255 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 64. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
I20240502 14:00:00.682790 28493 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 65, Time since last communication: 0.000s
I20240502 14:00:00.683125 28493 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 64, Time since last communication: 0.000s
I20240502 14:00:00.686017 28543 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [term 1 LEADER]: Committing config change with OpId 1.66: config changed from index -1 to 66, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.687546 28495 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 64, Time since last communication: 0.000s
I20240502 14:00:00.696661 26987 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index -1 to 66, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.696696 27253 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index -1 to 66, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.699855 26713 catalog_manager.cc:5484] T 6ed88ddf0d1b4dabbbf4604019705436 P b4fc3c795b1d4505b3a2903532d38b52 reported cstate change: config changed from index -1 to 66, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New cstate: current_term: 1 leader_uuid: "0280081e4ff74896832ec3985c372b3a" committed_config { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.709095 27518 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index -1 to 66, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.737577 27252 consensus_queue.cc:237] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 1.34, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:00.739343 26858 raft_consensus.cc:1261] T a734b1d6d928408cac2a9e1b263f2697 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 34. Preceding OpId from leader: term: 1 index: 35. (index mismatch)
I20240502 14:00:00.739367 27515 raft_consensus.cc:1261] T a734b1d6d928408cac2a9e1b263f2697 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 34. Preceding OpId from leader: term: 1 index: 35. (index mismatch)
W20240502 14:00:00.739593 27188 consensus_peers.cc:485] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 -> Peer 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71:42201): Couldn't send request to peer 5f9ddfa1f6fc484d8fb96075a8efb649. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a734b1d6d928408cac2a9e1b263f2697. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:00.739784 27385 raft_consensus.cc:1261] T a734b1d6d928408cac2a9e1b263f2697 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 34. Preceding OpId from leader: term: 1 index: 35. (index mismatch)
I20240502 14:00:00.739784 28680 consensus_queue.cc:1035] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20240502 14:00:00.740264 28680 consensus_queue.cc:1035] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20240502 14:00:00.740533 28681 consensus_queue.cc:1035] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20240502 14:00:00.742067 26985 raft_consensus.cc:1261] T a734b1d6d928408cac2a9e1b263f2697 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 34. Preceding OpId from leader: term: 1 index: 35. (index mismatch)
I20240502 14:00:00.742635 28681 consensus_queue.cc:1035] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 35, Last known committed idx: 34, Time since last communication: 0.000s
I20240502 14:00:00.743759 28681 raft_consensus.cc:2932] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 LEADER]: Committing config change with OpId 1.35: config changed from index -1 to 35, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.747161 26858 raft_consensus.cc:2932] T a734b1d6d928408cac2a9e1b263f2697 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.35: config changed from index -1 to 35, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.747810 26712 catalog_manager.cc:5484] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 reported cstate change: config changed from index -1 to 35, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New cstate: current_term: 1 leader_uuid: "b4fc3c795b1d4505b3a2903532d38b52" committed_config { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:00.752231 27385 raft_consensus.cc:2932] T a734b1d6d928408cac2a9e1b263f2697 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.35: config changed from index -1 to 35, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.755822 26985 raft_consensus.cc:2932] T a734b1d6d928408cac2a9e1b263f2697 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.35: config changed from index -1 to 35, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.757030 27519 raft_consensus.cc:2932] T a734b1d6d928408cac2a9e1b263f2697 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.35: config changed from index -1 to 35, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.774571 27386 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index -1 to 66, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.792312 26985 consensus_queue.cc:237] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 51, Committed index: 51, Last appended: 1.52, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:00.795246 26854 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 52. Preceding OpId from leader: term: 1 index: 53. (index mismatch)
I20240502 14:00:00.795357 27252 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 52. Preceding OpId from leader: term: 1 index: 53. (index mismatch)
I20240502 14:00:00.795982 28019 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 53, Last known committed idx: 51, Time since last communication: 0.000s
I20240502 14:00:00.796259 28019 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 53, Last known committed idx: 51, Time since last communication: 0.000s
I20240502 14:00:00.798840 28448 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 LEADER]: Committing config change with OpId 1.53: config changed from index -1 to 53, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.800098 27384 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 52. Preceding OpId from leader: term: 1 index: 53. (index mismatch)
I20240502 14:00:00.800911 26854 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.53: config changed from index -1 to 53, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.801941 27252 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.53: config changed from index -1 to 53, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.803180 28019 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 53, Last known committed idx: 51, Time since last communication: 0.000s
I20240502 14:00:00.804463 26713 catalog_manager.cc:5484] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68 reported cstate change: config changed from index -1 to 53, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New cstate: current_term: 1 leader_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" committed_config { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.804296 27384 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.53: config changed from index -1 to 53, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:00.805519 27122 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 52. Preceding OpId from leader: term: 1 index: 53. (index mismatch)
I20240502 14:00:00.810859 28449 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 53, Last known committed idx: 51, Time since last communication: 0.000s
I20240502 14:00:00.815137 27115 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.53: config changed from index -1 to 53, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
W20240502 14:00:00.820246 26924 consensus_peers.cc:485] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 -> Peer 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71:42201): Couldn't send request to peer 5f9ddfa1f6fc484d8fb96075a8efb649. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 41e73be38c4348a5bb86da6c0f6c6d22. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:00.911536 28724 ts_tablet_manager.cc:902] T 95f064e56ee241dba95f9a6cd5984d97 P 5f9ddfa1f6fc484d8fb96075a8efb649: Initiating tablet copy from peer b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68:35167)
I20240502 14:00:00.913012 28724 tablet_copy_client.cc:308] T 95f064e56ee241dba95f9a6cd5984d97 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:35167
I20240502 14:00:00.943480 28732 ts_tablet_manager.cc:902] T 4a74c0c312a7403d89958a3632cab7d3 P 256ce64303ec49e1a5cd8e0ffee436b0: Initiating tablet copy from peer 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733)
I20240502 14:00:00.943997 28731 ts_tablet_manager.cc:902] T 9efaa3c68c534606bd37bbe4703b2ab2 P 5f9ddfa1f6fc484d8fb96075a8efb649: Initiating tablet copy from peer 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217)
I20240502 14:00:00.944860 28731 tablet_copy_client.cc:308] T 9efaa3c68c534606bd37bbe4703b2ab2 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.70:38217
I20240502 14:00:00.946331 28732 tablet_copy_client.cc:308] T 4a74c0c312a7403d89958a3632cab7d3 P 256ce64303ec49e1a5cd8e0ffee436b0: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.67:35733
I20240502 14:00:00.947083 27144 tablet_copy_service.cc:140] P 0280081e4ff74896832ec3985c372b3a: Received BeginTabletCopySession request for tablet 4a74c0c312a7403d89958a3632cab7d3 from peer 256ce64303ec49e1a5cd8e0ffee436b0 ({username='slave'} at 127.24.66.66:47919)
I20240502 14:00:00.947218 27144 tablet_copy_service.cc:161] P 0280081e4ff74896832ec3985c372b3a: Beginning new tablet copy session on tablet 4a74c0c312a7403d89958a3632cab7d3 from peer 256ce64303ec49e1a5cd8e0ffee436b0 at {username='slave'} at 127.24.66.66:47919: session id = 256ce64303ec49e1a5cd8e0ffee436b0-4a74c0c312a7403d89958a3632cab7d3
I20240502 14:00:00.948873 27144 tablet_copy_source_session.cc:205] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:00.959558 28732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a74c0c312a7403d89958a3632cab7d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:00.971439 27275 tablet_copy_service.cc:140] P b4fc3c795b1d4505b3a2903532d38b52: Received BeginTabletCopySession request for tablet 95f064e56ee241dba95f9a6cd5984d97 from peer 5f9ddfa1f6fc484d8fb96075a8efb649 ({username='slave'} at 127.24.66.71:54201)
I20240502 14:00:00.971627 27275 tablet_copy_service.cc:161] P b4fc3c795b1d4505b3a2903532d38b52: Beginning new tablet copy session on tablet 95f064e56ee241dba95f9a6cd5984d97 from peer 5f9ddfa1f6fc484d8fb96075a8efb649 at {username='slave'} at 127.24.66.71:54201: session id = 5f9ddfa1f6fc484d8fb96075a8efb649-95f064e56ee241dba95f9a6cd5984d97
I20240502 14:00:00.973243 27275 tablet_copy_source_session.cc:205] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:00.975610 28724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95f064e56ee241dba95f9a6cd5984d97. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:00.982168 28724 tablet_copy_client.cc:783] T 95f064e56ee241dba95f9a6cd5984d97 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:00.982439 28724 tablet_copy_client.cc:647] T 95f064e56ee241dba95f9a6cd5984d97 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:00.986332 27540 tablet_copy_service.cc:140] P 34b161bd719d4d72baf8da4f40aef136: Received BeginTabletCopySession request for tablet 9efaa3c68c534606bd37bbe4703b2ab2 from peer 5f9ddfa1f6fc484d8fb96075a8efb649 ({username='slave'} at 127.24.66.71:46263)
I20240502 14:00:00.986505 27540 tablet_copy_service.cc:161] P 34b161bd719d4d72baf8da4f40aef136: Beginning new tablet copy session on tablet 9efaa3c68c534606bd37bbe4703b2ab2 from peer 5f9ddfa1f6fc484d8fb96075a8efb649 at {username='slave'} at 127.24.66.71:46263: session id = 5f9ddfa1f6fc484d8fb96075a8efb649-9efaa3c68c534606bd37bbe4703b2ab2
I20240502 14:00:00.987908 27540 tablet_copy_source_session.cc:205] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:00.991159 28724 tablet_copy_client.cc:523] T 95f064e56ee241dba95f9a6cd5984d97 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:00.992904 28724 tablet_bootstrap.cc:492] T 95f064e56ee241dba95f9a6cd5984d97 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap starting.
I20240502 14:00:00.997283 28731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9efaa3c68c534606bd37bbe4703b2ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:01.026985 28732 tablet_copy_client.cc:783] T 4a74c0c312a7403d89958a3632cab7d3 P 256ce64303ec49e1a5cd8e0ffee436b0: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:01.027295 28732 tablet_copy_client.cc:647] T 4a74c0c312a7403d89958a3632cab7d3 P 256ce64303ec49e1a5cd8e0ffee436b0: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:01.028091 28731 tablet_copy_client.cc:783] T 9efaa3c68c534606bd37bbe4703b2ab2 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:01.028334 28731 tablet_copy_client.cc:647] T 9efaa3c68c534606bd37bbe4703b2ab2 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:01.037930 28731 tablet_copy_client.cc:523] T 9efaa3c68c534606bd37bbe4703b2ab2 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:01.039467 28731 tablet_bootstrap.cc:492] T 9efaa3c68c534606bd37bbe4703b2ab2 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap starting.
I20240502 14:00:01.057812 28732 tablet_copy_client.cc:523] T 4a74c0c312a7403d89958a3632cab7d3 P 256ce64303ec49e1a5cd8e0ffee436b0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:01.061000 28732 tablet_bootstrap.cc:492] T 4a74c0c312a7403d89958a3632cab7d3 P 256ce64303ec49e1a5cd8e0ffee436b0: Bootstrap starting.
I20240502 14:00:01.147979 28761 ts_tablet_manager.cc:902] T 6ed88ddf0d1b4dabbbf4604019705436 P 5f9ddfa1f6fc484d8fb96075a8efb649: Initiating tablet copy from peer 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733)
I20240502 14:00:01.149588 28761 tablet_copy_client.cc:308] T 6ed88ddf0d1b4dabbbf4604019705436 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.67:35733
I20240502 14:00:01.158885 27144 tablet_copy_service.cc:140] P 0280081e4ff74896832ec3985c372b3a: Received BeginTabletCopySession request for tablet 6ed88ddf0d1b4dabbbf4604019705436 from peer 5f9ddfa1f6fc484d8fb96075a8efb649 ({username='slave'} at 127.24.66.71:36675)
I20240502 14:00:01.159018 27144 tablet_copy_service.cc:161] P 0280081e4ff74896832ec3985c372b3a: Beginning new tablet copy session on tablet 6ed88ddf0d1b4dabbbf4604019705436 from peer 5f9ddfa1f6fc484d8fb96075a8efb649 at {username='slave'} at 127.24.66.71:36675: session id = 5f9ddfa1f6fc484d8fb96075a8efb649-6ed88ddf0d1b4dabbbf4604019705436
I20240502 14:00:01.159987 27144 tablet_copy_source_session.cc:205] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:01.176494 28761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ed88ddf0d1b4dabbbf4604019705436. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:01.180269 28769 ts_tablet_manager.cc:902] T 41e73be38c4348a5bb86da6c0f6c6d22 P 5f9ddfa1f6fc484d8fb96075a8efb649: Initiating tablet copy from peer 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473)
I20240502 14:00:01.181425 28769 tablet_copy_client.cc:308] T 41e73be38c4348a5bb86da6c0f6c6d22 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.66:40473
I20240502 14:00:01.201063 28761 tablet_copy_client.cc:783] T 6ed88ddf0d1b4dabbbf4604019705436 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:01.201256 28761 tablet_copy_client.cc:647] T 6ed88ddf0d1b4dabbbf4604019705436 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:01.206732 27012 tablet_copy_service.cc:140] P 256ce64303ec49e1a5cd8e0ffee436b0: Received BeginTabletCopySession request for tablet 41e73be38c4348a5bb86da6c0f6c6d22 from peer 5f9ddfa1f6fc484d8fb96075a8efb649 ({username='slave'} at 127.24.66.71:42147)
I20240502 14:00:01.206890 27012 tablet_copy_service.cc:161] P 256ce64303ec49e1a5cd8e0ffee436b0: Beginning new tablet copy session on tablet 41e73be38c4348a5bb86da6c0f6c6d22 from peer 5f9ddfa1f6fc484d8fb96075a8efb649 at {username='slave'} at 127.24.66.71:42147: session id = 5f9ddfa1f6fc484d8fb96075a8efb649-41e73be38c4348a5bb86da6c0f6c6d22
I20240502 14:00:01.208221 27012 tablet_copy_source_session.cc:205] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:01.217550 28775 ts_tablet_manager.cc:902] T a734b1d6d928408cac2a9e1b263f2697 P 5f9ddfa1f6fc484d8fb96075a8efb649: Initiating tablet copy from peer b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68:35167)
I20240502 14:00:01.218215 28775 tablet_copy_client.cc:308] T a734b1d6d928408cac2a9e1b263f2697 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:35167
I20240502 14:00:01.218766 28761 tablet_copy_client.cc:523] T 6ed88ddf0d1b4dabbbf4604019705436 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:01.219875 27275 tablet_copy_service.cc:140] P b4fc3c795b1d4505b3a2903532d38b52: Received BeginTabletCopySession request for tablet a734b1d6d928408cac2a9e1b263f2697 from peer 5f9ddfa1f6fc484d8fb96075a8efb649 ({username='slave'} at 127.24.66.71:54201)
I20240502 14:00:01.219987 27275 tablet_copy_service.cc:161] P b4fc3c795b1d4505b3a2903532d38b52: Beginning new tablet copy session on tablet a734b1d6d928408cac2a9e1b263f2697 from peer 5f9ddfa1f6fc484d8fb96075a8efb649 at {username='slave'} at 127.24.66.71:54201: session id = 5f9ddfa1f6fc484d8fb96075a8efb649-a734b1d6d928408cac2a9e1b263f2697
I20240502 14:00:01.220480 28769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41e73be38c4348a5bb86da6c0f6c6d22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:01.221025 27275 tablet_copy_source_session.cc:205] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:01.222496 28769 tablet_copy_client.cc:783] T 41e73be38c4348a5bb86da6c0f6c6d22 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:01.222664 28769 tablet_copy_client.cc:647] T 41e73be38c4348a5bb86da6c0f6c6d22 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:01.226394 28769 tablet_copy_client.cc:523] T 41e73be38c4348a5bb86da6c0f6c6d22 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:01.227670 28769 tablet_bootstrap.cc:492] T 41e73be38c4348a5bb86da6c0f6c6d22 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap starting.
I20240502 14:00:01.244709 28775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a734b1d6d928408cac2a9e1b263f2697. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:01.249341 28761 tablet_bootstrap.cc:492] T 6ed88ddf0d1b4dabbbf4604019705436 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap starting.
I20240502 14:00:01.290946 28775 tablet_copy_client.cc:783] T a734b1d6d928408cac2a9e1b263f2697 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:01.291416 28775 tablet_copy_client.cc:647] T a734b1d6d928408cac2a9e1b263f2697 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:01.298312 28775 tablet_copy_client.cc:523] T a734b1d6d928408cac2a9e1b263f2697 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:01.306134 28775 tablet_bootstrap.cc:492] T a734b1d6d928408cac2a9e1b263f2697 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap starting.
W20240502 14:00:01.306886 26927 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.66.68:35167, user_credentials={real_user=slave}} blocked reactor thread for 49204.1us
W20240502 14:00:01.527894 27056 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.66.66:40473, user_credentials={real_user=slave}} blocked reactor thread for 53830.7us
I20240502 14:00:01.595746 28724 tablet_bootstrap.cc:492] T 95f064e56ee241dba95f9a6cd5984d97 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap replayed 1/1 log segments. Stats: ops{read=49 overwritten=0 applied=49 ignored=0} inserts{seen=680 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:01.600718 28724 tablet_bootstrap.cc:492] T 95f064e56ee241dba95f9a6cd5984d97 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap complete.
I20240502 14:00:01.603021 28724 ts_tablet_manager.cc:1367] T 95f064e56ee241dba95f9a6cd5984d97 P 5f9ddfa1f6fc484d8fb96075a8efb649: Time spent bootstrapping tablet: real 0.610s user 0.059s sys 0.008s
I20240502 14:00:01.607223 28724 raft_consensus.cc:348] T 95f064e56ee241dba95f9a6cd5984d97 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:01.611732 28724 raft_consensus.cc:729] T 95f064e56ee241dba95f9a6cd5984d97 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f9ddfa1f6fc484d8fb96075a8efb649, State: Initialized, Role: LEARNER
I20240502 14:00:01.625945 28724 consensus_queue.cc:260] T 95f064e56ee241dba95f9a6cd5984d97 P 5f9ddfa1f6fc484d8fb96075a8efb649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 49, Last appended: 1.49, Last appended by leader: 49, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:01.628016 28724 ts_tablet_manager.cc:1398] T 95f064e56ee241dba95f9a6cd5984d97 P 5f9ddfa1f6fc484d8fb96075a8efb649: Time spent starting tablet: real 0.021s user 0.001s sys 0.000s
I20240502 14:00:01.630738 27275 tablet_copy_service.cc:342] P b4fc3c795b1d4505b3a2903532d38b52: Request end of tablet copy session 5f9ddfa1f6fc484d8fb96075a8efb649-95f064e56ee241dba95f9a6cd5984d97 received from {username='slave'} at 127.24.66.71:54201
I20240502 14:00:01.630861 27275 tablet_copy_service.cc:433] P b4fc3c795b1d4505b3a2903532d38b52: ending tablet copy session 5f9ddfa1f6fc484d8fb96075a8efb649-95f064e56ee241dba95f9a6cd5984d97 on tablet 95f064e56ee241dba95f9a6cd5984d97 with peer 5f9ddfa1f6fc484d8fb96075a8efb649
I20240502 14:00:01.746168 28775 tablet_bootstrap.cc:492] T a734b1d6d928408cac2a9e1b263f2697 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap replayed 1/1 log segments. Stats: ops{read=42 overwritten=0 applied=42 ignored=0} inserts{seen=714 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:01.750267 28775 tablet_bootstrap.cc:492] T a734b1d6d928408cac2a9e1b263f2697 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap complete.
I20240502 14:00:01.751694 28775 ts_tablet_manager.cc:1367] T a734b1d6d928408cac2a9e1b263f2697 P 5f9ddfa1f6fc484d8fb96075a8efb649: Time spent bootstrapping tablet: real 0.446s user 0.056s sys 0.000s
I20240502 14:00:01.760833 28775 raft_consensus.cc:348] T a734b1d6d928408cac2a9e1b263f2697 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:01.761081 28775 raft_consensus.cc:729] T a734b1d6d928408cac2a9e1b263f2697 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f9ddfa1f6fc484d8fb96075a8efb649, State: Initialized, Role: LEARNER
I20240502 14:00:01.761247 28775 consensus_queue.cc:260] T a734b1d6d928408cac2a9e1b263f2697 P 5f9ddfa1f6fc484d8fb96075a8efb649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 42, Last appended: 1.42, Last appended by leader: 42, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 35 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:01.762406 28775 ts_tablet_manager.cc:1398] T a734b1d6d928408cac2a9e1b263f2697 P 5f9ddfa1f6fc484d8fb96075a8efb649: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:01.763465 27275 tablet_copy_service.cc:342] P b4fc3c795b1d4505b3a2903532d38b52: Request end of tablet copy session 5f9ddfa1f6fc484d8fb96075a8efb649-a734b1d6d928408cac2a9e1b263f2697 received from {username='slave'} at 127.24.66.71:54201
I20240502 14:00:01.763561 27275 tablet_copy_service.cc:433] P b4fc3c795b1d4505b3a2903532d38b52: ending tablet copy session 5f9ddfa1f6fc484d8fb96075a8efb649-a734b1d6d928408cac2a9e1b263f2697 on tablet a734b1d6d928408cac2a9e1b263f2697 with peer 5f9ddfa1f6fc484d8fb96075a8efb649
I20240502 14:00:01.764072 28731 tablet_bootstrap.cc:492] T 9efaa3c68c534606bd37bbe4703b2ab2 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap replayed 1/1 log segments. Stats: ops{read=68 overwritten=0 applied=68 ignored=0} inserts{seen=753 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:01.764448 28731 tablet_bootstrap.cc:492] T 9efaa3c68c534606bd37bbe4703b2ab2 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap complete.
I20240502 14:00:01.764670 28731 ts_tablet_manager.cc:1367] T 9efaa3c68c534606bd37bbe4703b2ab2 P 5f9ddfa1f6fc484d8fb96075a8efb649: Time spent bootstrapping tablet: real 0.725s user 0.068s sys 0.009s
I20240502 14:00:01.765084 28731 raft_consensus.cc:348] T 9efaa3c68c534606bd37bbe4703b2ab2 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:01.765869 28731 raft_consensus.cc:729] T 9efaa3c68c534606bd37bbe4703b2ab2 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f9ddfa1f6fc484d8fb96075a8efb649, State: Initialized, Role: LEARNER
I20240502 14:00:01.766026 28731 consensus_queue.cc:260] T 9efaa3c68c534606bd37bbe4703b2ab2 P 5f9ddfa1f6fc484d8fb96075a8efb649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 68, Last appended: 1.68, Last appended by leader: 68, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 56 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:01.766397 28731 ts_tablet_manager.cc:1398] T 9efaa3c68c534606bd37bbe4703b2ab2 P 5f9ddfa1f6fc484d8fb96075a8efb649: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:01.766880 27540 tablet_copy_service.cc:342] P 34b161bd719d4d72baf8da4f40aef136: Request end of tablet copy session 5f9ddfa1f6fc484d8fb96075a8efb649-9efaa3c68c534606bd37bbe4703b2ab2 received from {username='slave'} at 127.24.66.71:46263
I20240502 14:00:01.766961 27540 tablet_copy_service.cc:433] P 34b161bd719d4d72baf8da4f40aef136: ending tablet copy session 5f9ddfa1f6fc484d8fb96075a8efb649-9efaa3c68c534606bd37bbe4703b2ab2 on tablet 9efaa3c68c534606bd37bbe4703b2ab2 with peer 5f9ddfa1f6fc484d8fb96075a8efb649
I20240502 14:00:01.774359 28732 tablet_bootstrap.cc:492] T 4a74c0c312a7403d89958a3632cab7d3 P 256ce64303ec49e1a5cd8e0ffee436b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=53 overwritten=0 applied=53 ignored=0} inserts{seen=738 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:01.776000 28732 tablet_bootstrap.cc:492] T 4a74c0c312a7403d89958a3632cab7d3 P 256ce64303ec49e1a5cd8e0ffee436b0: Bootstrap complete.
I20240502 14:00:01.776309 28732 ts_tablet_manager.cc:1367] T 4a74c0c312a7403d89958a3632cab7d3 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent bootstrapping tablet: real 0.715s user 0.068s sys 0.000s
I20240502 14:00:01.778189 28732 raft_consensus.cc:348] T 4a74c0c312a7403d89958a3632cab7d3 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 50 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: true } }
I20240502 14:00:01.778384 28732 raft_consensus.cc:729] T 4a74c0c312a7403d89958a3632cab7d3 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 256ce64303ec49e1a5cd8e0ffee436b0, State: Initialized, Role: LEARNER
I20240502 14:00:01.778537 28732 consensus_queue.cc:260] T 4a74c0c312a7403d89958a3632cab7d3 P 256ce64303ec49e1a5cd8e0ffee436b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 53, Last appended: 1.53, Last appended by leader: 53, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 50 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: true } }
I20240502 14:00:01.755167 28769 tablet_bootstrap.cc:492] T 41e73be38c4348a5bb86da6c0f6c6d22 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap replayed 1/1 log segments. Stats: ops{read=63 overwritten=0 applied=63 ignored=0} inserts{seen=735 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:01.783942 28732 ts_tablet_manager.cc:1398] T 4a74c0c312a7403d89958a3632cab7d3 P 256ce64303ec49e1a5cd8e0ffee436b0: Time spent starting tablet: real 0.006s user 0.000s sys 0.001s
I20240502 14:00:01.784718 28769 tablet_bootstrap.cc:492] T 41e73be38c4348a5bb86da6c0f6c6d22 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap complete.
I20240502 14:00:01.784977 28769 ts_tablet_manager.cc:1367] T 41e73be38c4348a5bb86da6c0f6c6d22 P 5f9ddfa1f6fc484d8fb96075a8efb649: Time spent bootstrapping tablet: real 0.557s user 0.068s sys 0.003s
I20240502 14:00:01.786757 28769 raft_consensus.cc:348] T 41e73be38c4348a5bb86da6c0f6c6d22 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:01.786947 28769 raft_consensus.cc:729] T 41e73be38c4348a5bb86da6c0f6c6d22 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f9ddfa1f6fc484d8fb96075a8efb649, State: Initialized, Role: LEARNER
I20240502 14:00:01.787103 28769 consensus_queue.cc:260] T 41e73be38c4348a5bb86da6c0f6c6d22 P 5f9ddfa1f6fc484d8fb96075a8efb649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 63, Last appended: 1.63, Last appended by leader: 63, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:01.788070 28769 ts_tablet_manager.cc:1398] T 41e73be38c4348a5bb86da6c0f6c6d22 P 5f9ddfa1f6fc484d8fb96075a8efb649: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:01.792062 27012 tablet_copy_service.cc:342] P 256ce64303ec49e1a5cd8e0ffee436b0: Request end of tablet copy session 5f9ddfa1f6fc484d8fb96075a8efb649-41e73be38c4348a5bb86da6c0f6c6d22 received from {username='slave'} at 127.24.66.71:42147
I20240502 14:00:01.792166 27012 tablet_copy_service.cc:433] P 256ce64303ec49e1a5cd8e0ffee436b0: ending tablet copy session 5f9ddfa1f6fc484d8fb96075a8efb649-41e73be38c4348a5bb86da6c0f6c6d22 on tablet 41e73be38c4348a5bb86da6c0f6c6d22 with peer 5f9ddfa1f6fc484d8fb96075a8efb649
I20240502 14:00:01.794649 27144 tablet_copy_service.cc:342] P 0280081e4ff74896832ec3985c372b3a: Request end of tablet copy session 256ce64303ec49e1a5cd8e0ffee436b0-4a74c0c312a7403d89958a3632cab7d3 received from {username='slave'} at 127.24.66.66:47919
I20240502 14:00:01.794749 27144 tablet_copy_service.cc:433] P 0280081e4ff74896832ec3985c372b3a: ending tablet copy session 256ce64303ec49e1a5cd8e0ffee436b0-4a74c0c312a7403d89958a3632cab7d3 on tablet 4a74c0c312a7403d89958a3632cab7d3 with peer 256ce64303ec49e1a5cd8e0ffee436b0
I20240502 14:00:01.816629 28365 raft_consensus.cc:1203] T a734b1d6d928408cac2a9e1b263f2697 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.34->[1.35-1.47] Dedup: 1.42->[1.43-1.47]
I20240502 14:00:01.816939 28761 tablet_bootstrap.cc:492] T 6ed88ddf0d1b4dabbbf4604019705436 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap replayed 1/1 log segments. Stats: ops{read=79 overwritten=0 applied=78 ignored=0} inserts{seen=709 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 14:00:01.817282 28761 tablet_bootstrap.cc:492] T 6ed88ddf0d1b4dabbbf4604019705436 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap complete.
I20240502 14:00:01.817484 28761 ts_tablet_manager.cc:1367] T 6ed88ddf0d1b4dabbbf4604019705436 P 5f9ddfa1f6fc484d8fb96075a8efb649: Time spent bootstrapping tablet: real 0.569s user 0.070s sys 0.008s
I20240502 14:00:01.817867 28761 raft_consensus.cc:348] T 6ed88ddf0d1b4dabbbf4604019705436 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:01.818219 28761 raft_consensus.cc:729] T 6ed88ddf0d1b4dabbbf4604019705436 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f9ddfa1f6fc484d8fb96075a8efb649, State: Initialized, Role: LEARNER
I20240502 14:00:01.818361 28761 consensus_queue.cc:260] T 6ed88ddf0d1b4dabbbf4604019705436 P 5f9ddfa1f6fc484d8fb96075a8efb649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 78, Last appended: 1.79, Last appended by leader: 79, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:01.818670 28761 ts_tablet_manager.cc:1398] T 6ed88ddf0d1b4dabbbf4604019705436 P 5f9ddfa1f6fc484d8fb96075a8efb649: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:00:01.824637 27144 tablet_copy_service.cc:342] P 0280081e4ff74896832ec3985c372b3a: Request end of tablet copy session 5f9ddfa1f6fc484d8fb96075a8efb649-6ed88ddf0d1b4dabbbf4604019705436 received from {username='slave'} at 127.24.66.71:36675
I20240502 14:00:01.824734 27144 tablet_copy_service.cc:433] P 0280081e4ff74896832ec3985c372b3a: ending tablet copy session 5f9ddfa1f6fc484d8fb96075a8efb649-6ed88ddf0d1b4dabbbf4604019705436 on tablet 6ed88ddf0d1b4dabbbf4604019705436 with peer 5f9ddfa1f6fc484d8fb96075a8efb649
I20240502 14:00:01.849943 28364 raft_consensus.cc:1203] T 95f064e56ee241dba95f9a6cd5984d97 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.44->[1.45-1.61] Dedup: 1.49->[1.50-1.61]
I20240502 14:00:01.901333 28500 raft_consensus.cc:1050] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52: attempting to promote NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 to VOTER
I20240502 14:00:01.902041 28500 consensus_queue.cc:237] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 62, Committed index: 62, Last appended: 1.62, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 63 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } }
I20240502 14:00:01.904543 28364 raft_consensus.cc:1261] T 95f064e56ee241dba95f9a6cd5984d97 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 62. Preceding OpId from leader: term: 1 index: 63. (index mismatch)
I20240502 14:00:01.904742 27515 raft_consensus.cc:1261] T 95f064e56ee241dba95f9a6cd5984d97 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 62. Preceding OpId from leader: term: 1 index: 63. (index mismatch)
I20240502 14:00:01.904951 26854 raft_consensus.cc:1261] T 95f064e56ee241dba95f9a6cd5984d97 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 62. Preceding OpId from leader: term: 1 index: 63. (index mismatch)
I20240502 14:00:01.905339 28500 consensus_queue.cc:1035] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 63, Last known committed idx: 62, Time since last communication: 0.000s
I20240502 14:00:01.905669 28617 consensus_queue.cc:1035] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 63, Last known committed idx: 62, Time since last communication: 0.000s
I20240502 14:00:01.907311 28680 consensus_queue.cc:1035] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 63, Last known committed idx: 62, Time since last communication: 0.000s
I20240502 14:00:01.908586 27387 raft_consensus.cc:1261] T 95f064e56ee241dba95f9a6cd5984d97 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 62. Preceding OpId from leader: term: 1 index: 63. (index mismatch)
I20240502 14:00:01.911784 28048 consensus_queue.cc:1035] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 63, Last known committed idx: 61, Time since last communication: 0.000s
I20240502 14:00:01.933418 28531 raft_consensus.cc:2932] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 LEADER]: Committing config change with OpId 1.63: config changed from index 45 to 63, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 63 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:01.937836 27385 raft_consensus.cc:2932] T 95f064e56ee241dba95f9a6cd5984d97 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.63: config changed from index 45 to 63, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 63 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:01.938151 26709 catalog_manager.cc:5484] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 reported cstate change: config changed from index 45 to 63, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b4fc3c795b1d4505b3a2903532d38b52" committed_config { opid_index: 63 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:01.934525 28365 raft_consensus.cc:2932] T 95f064e56ee241dba95f9a6cd5984d97 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.63: config changed from index 45 to 63, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 63 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:01.941064 26856 raft_consensus.cc:2932] T 95f064e56ee241dba95f9a6cd5984d97 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.63: config changed from index 45 to 63, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 63 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:01.946424 27515 raft_consensus.cc:2932] T 95f064e56ee241dba95f9a6cd5984d97 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.63: config changed from index 45 to 63, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 63 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:01.965646 28680 raft_consensus.cc:1050] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52: attempting to promote NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 to VOTER
I20240502 14:00:01.966239 28680 consensus_queue.cc:237] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 48, Committed index: 48, Last appended: 1.48, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } }
I20240502 14:00:01.969425 26855 raft_consensus.cc:1261] T a734b1d6d928408cac2a9e1b263f2697 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 48. Preceding OpId from leader: term: 1 index: 49. (index mismatch)
I20240502 14:00:01.970019 28363 raft_consensus.cc:1261] T a734b1d6d928408cac2a9e1b263f2697 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 48. Preceding OpId from leader: term: 1 index: 49. (index mismatch)
I20240502 14:00:01.970245 27518 raft_consensus.cc:1261] T a734b1d6d928408cac2a9e1b263f2697 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 48. Preceding OpId from leader: term: 1 index: 49. (index mismatch)
I20240502 14:00:01.972787 28680 consensus_queue.cc:1035] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 49, Last known committed idx: 48, Time since last communication: 0.000s
I20240502 14:00:01.974712 28680 consensus_queue.cc:1035] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 49, Last known committed idx: 48, Time since last communication: 0.000s
I20240502 14:00:01.978771 27118 raft_consensus.cc:1261] T 95f064e56ee241dba95f9a6cd5984d97 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 62. Preceding OpId from leader: term: 1 index: 63. (index mismatch)
I20240502 14:00:01.979375 28617 consensus_queue.cc:1035] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 63, Last known committed idx: 62, Time since last communication: 0.000s
I20240502 14:00:01.981547 27122 raft_consensus.cc:2932] T 95f064e56ee241dba95f9a6cd5984d97 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.63: config changed from index 45 to 63, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 63 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:01.989408 28617 consensus_queue.cc:1035] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 49, Last known committed idx: 47, Time since last communication: 0.000s
I20240502 14:00:01.996536 27386 raft_consensus.cc:1261] T a734b1d6d928408cac2a9e1b263f2697 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 48. Preceding OpId from leader: term: 1 index: 49. (index mismatch)
I20240502 14:00:01.997715 28717 consensus_queue.cc:1035] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 49, Last known committed idx: 48, Time since last communication: 0.000s
I20240502 14:00:02.005925 28680 raft_consensus.cc:2932] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 LEADER]: Committing config change with OpId 1.49: config changed from index 35 to 49, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.012187 27386 raft_consensus.cc:2932] T a734b1d6d928408cac2a9e1b263f2697 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.49: config changed from index 35 to 49, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
W20240502 14:00:02.025553 26927 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.66.67:35733, user_credentials={real_user=slave}} blocked reactor thread for 61656.1us
I20240502 14:00:02.027806 26856 raft_consensus.cc:2932] T a734b1d6d928408cac2a9e1b263f2697 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.49: config changed from index 35 to 49, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.061100 27515 raft_consensus.cc:2932] T a734b1d6d928408cac2a9e1b263f2697 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.49: config changed from index 35 to 49, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.069003 26985 raft_consensus.cc:1203] T 4a74c0c312a7403d89958a3632cab7d3 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.49->[1.50-1.61] Dedup: 1.53->[1.54-1.61]
I20240502 14:00:02.072986 28365 raft_consensus.cc:2932] T a734b1d6d928408cac2a9e1b263f2697 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.49: config changed from index 35 to 49, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.076951 26991 raft_consensus.cc:1261] T a734b1d6d928408cac2a9e1b263f2697 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 48. Preceding OpId from leader: term: 1 index: 49. (index mismatch)
I20240502 14:00:02.173043 28832 consensus_queue.cc:1035] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 49, Last known committed idx: 48, Time since last communication: 0.000s
I20240502 14:00:02.196827 26712 catalog_manager.cc:5484] T a734b1d6d928408cac2a9e1b263f2697 P e44f8c16a14a48a38a5a74af2899c217 reported cstate change: config changed from index 35 to 49, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b4fc3c795b1d4505b3a2903532d38b52" committed_config { opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.204208 27252 consensus_queue.cc:237] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 65, Committed index: 65, Last appended: 1.65, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } }
I20240502 14:00:02.214190 27384 raft_consensus.cc:1261] T 95f064e56ee241dba95f9a6cd5984d97 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 66. (index mismatch)
I20240502 14:00:02.214530 28832 consensus_queue.cc:1035] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 65, Time since last communication: 0.000s
I20240502 14:00:02.218420 26856 raft_consensus.cc:1261] T 95f064e56ee241dba95f9a6cd5984d97 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 66. (index mismatch)
I20240502 14:00:02.218485 27515 raft_consensus.cc:1261] T 95f064e56ee241dba95f9a6cd5984d97 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 66. (index mismatch)
I20240502 14:00:02.218590 28365 raft_consensus.cc:1261] T 95f064e56ee241dba95f9a6cd5984d97 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 66. (index mismatch)
I20240502 14:00:02.246943 28169 consensus_queue.cc:1035] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 65, Time since last communication: 0.000s
I20240502 14:00:02.247200 28531 consensus_queue.cc:1035] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 65, Time since last communication: 0.000s
I20240502 14:00:02.248301 28680 consensus_queue.cc:1035] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 65, Time since last communication: 0.000s
I20240502 14:00:02.261276 28365 raft_consensus.cc:1203] T 9efaa3c68c534606bd37bbe4703b2ab2 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.55->[1.56-1.84] Dedup: 1.68->[1.69-1.84]
I20240502 14:00:02.280802 28797 raft_consensus.cc:1050] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136: attempting to promote NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 to VOTER
I20240502 14:00:02.281351 28797 consensus_queue.cc:237] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 84, Committed index: 84, Last appended: 1.84, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 85 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } }
I20240502 14:00:02.281450 26987 raft_consensus.cc:2932] T a734b1d6d928408cac2a9e1b263f2697 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.49: config changed from index 35 to 49, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 49 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.282322 28708 raft_consensus.cc:1050] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a: attempting to promote NON_VOTER 256ce64303ec49e1a5cd8e0ffee436b0 to VOTER
I20240502 14:00:02.282824 28708 consensus_queue.cc:237] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 65, Committed index: 65, Last appended: 1.65, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } }
I20240502 14:00:02.284690 28365 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 84. Preceding OpId from leader: term: 1 index: 85. (index mismatch)
I20240502 14:00:02.285072 28614 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 85, Last known committed idx: 84, Time since last communication: 0.000s
I20240502 14:00:02.286084 27385 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 84. Preceding OpId from leader: term: 1 index: 85. (index mismatch)
I20240502 14:00:02.286458 28614 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 85, Last known committed idx: 84, Time since last communication: 0.000s
I20240502 14:00:02.288937 27123 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 84. Preceding OpId from leader: term: 1 index: 85. (index mismatch)
I20240502 14:00:02.289433 28615 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 85, Last known committed idx: 84, Time since last communication: 0.000s
I20240502 14:00:02.292222 27386 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 66. (index mismatch)
I20240502 14:00:02.292984 28495 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 65, Time since last communication: 0.000s
I20240502 14:00:02.293309 28364 raft_consensus.cc:1203] T 6ed88ddf0d1b4dabbbf4604019705436 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.65->[1.66-1.103] Dedup: 1.79->[1.80-1.103]
I20240502 14:00:02.307142 27255 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 64. Preceding OpId from leader: term: 1 index: 66. (index mismatch)
I20240502 14:00:02.310127 26856 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 66. (index mismatch)
I20240502 14:00:02.310633 28543 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 65, Time since last communication: 0.000s
I20240502 14:00:02.312273 28707 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 63, Time since last communication: 0.000s
I20240502 14:00:02.312386 26990 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 LEARNER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 66. (index mismatch)
I20240502 14:00:02.313421 26985 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 83. Preceding OpId from leader: term: 1 index: 85. (index mismatch)
I20240502 14:00:02.314301 28707 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 65, Time since last communication: 0.000s
I20240502 14:00:02.315238 28832 raft_consensus.cc:2932] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 LEADER]: Committing config change with OpId 1.66: config changed from index 63 to 66, VOTER 0280081e4ff74896832ec3985c372b3a (127.24.66.67) evicted. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.324637 27515 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 65. Preceding OpId from leader: term: 1 index: 66. (index mismatch)
I20240502 14:00:02.324995 28543 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 66, Last known committed idx: 65, Time since last communication: 0.000s
I20240502 14:00:02.326473 28543 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [term 1 LEADER]: Committing config change with OpId 1.66: config changed from index 50 to 66, 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) changed from NON_VOTER to VOTER. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } }
I20240502 14:00:02.328409 27517 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index 50 to 66, 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) changed from NON_VOTER to VOTER. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } }
I20240502 14:00:02.328604 28615 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [term 1 LEADER]: Committing config change with OpId 1.85: config changed from index 56 to 85, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 85 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.330195 28365 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.85: config changed from index 56 to 85, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 85 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.348158 27255 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 83. Preceding OpId from leader: term: 1 index: 85. (index mismatch)
I20240502 14:00:02.348757 28614 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 85, Last known committed idx: 83, Time since last communication: 0.000s
I20240502 14:00:02.357923 26714 catalog_manager.cc:5484] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136 reported cstate change: config changed from index 50 to 66, 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0280081e4ff74896832ec3985c372b3a" committed_config { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } }
I20240502 14:00:02.359565 26703 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 95f064e56ee241dba95f9a6cd5984d97 with cas_config_opid_index 63: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:02.360023 26710 catalog_manager.cc:5484] T 95f064e56ee241dba95f9a6cd5984d97 P b4fc3c795b1d4505b3a2903532d38b52 reported cstate change: config changed from index 63 to 66, VOTER 0280081e4ff74896832ec3985c372b3a (127.24.66.67) evicted. New cstate: current_term: 1 leader_uuid: "b4fc3c795b1d4505b3a2903532d38b52" committed_config { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:02.360365 26859 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index 50 to 66, 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) changed from NON_VOTER to VOTER. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } }
I20240502 14:00:02.361523 26854 raft_consensus.cc:2932] T 95f064e56ee241dba95f9a6cd5984d97 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index 63 to 66, VOTER 0280081e4ff74896832ec3985c372b3a (127.24.66.67) evicted. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.362584 28363 raft_consensus.cc:2932] T 95f064e56ee241dba95f9a6cd5984d97 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index 63 to 66, VOTER 0280081e4ff74896832ec3985c372b3a (127.24.66.67) evicted. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.363389 27101 tablet_service.cc:1514] Processing DeleteTablet for tablet 95f064e56ee241dba95f9a6cd5984d97 with delete_type TABLET_DATA_TOMBSTONED (TS 0280081e4ff74896832ec3985c372b3a not found in new config with opid_index 66) from {username='slave'} at 127.0.0.1:51610
I20240502 14:00:02.365342 26714 catalog_manager.cc:5484] T 9efaa3c68c534606bd37bbe4703b2ab2 P 5f9ddfa1f6fc484d8fb96075a8efb649 reported cstate change: config changed from index 56 to 85, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34b161bd719d4d72baf8da4f40aef136" committed_config { opid_index: 85 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.366098 27384 raft_consensus.cc:2932] T 95f064e56ee241dba95f9a6cd5984d97 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index 63 to 66, VOTER 0280081e4ff74896832ec3985c372b3a (127.24.66.67) evicted. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.368667 27386 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index 50 to 66, 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) changed from NON_VOTER to VOTER. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } }
I20240502 14:00:02.372144 28864 tablet_replica.cc:330] T 95f064e56ee241dba95f9a6cd5984d97 P 0280081e4ff74896832ec3985c372b3a: stopping tablet replica
I20240502 14:00:02.374037 28864 raft_consensus.cc:2229] T 95f064e56ee241dba95f9a6cd5984d97 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:02.374940 27519 raft_consensus.cc:2932] T 95f064e56ee241dba95f9a6cd5984d97 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index 63 to 66, VOTER 0280081e4ff74896832ec3985c372b3a (127.24.66.67) evicted. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.375118 28861 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 85, Last known committed idx: 83, Time since last communication: 0.000s
I20240502 14:00:02.378749 28864 raft_consensus.cc:2258] T 95f064e56ee241dba95f9a6cd5984d97 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:02.380263 28864 ts_tablet_manager.cc:1875] T 95f064e56ee241dba95f9a6cd5984d97 P 0280081e4ff74896832ec3985c372b3a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:02.387554 26987 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.85: config changed from index 56 to 85, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 85 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.402503 27123 consensus_queue.cc:237] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 66, Committed index: 66, Last appended: 1.66, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } }
I20240502 14:00:02.404450 27515 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 66. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
I20240502 14:00:02.404858 28495 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 66, Time since last communication: 0.000s
I20240502 14:00:02.407433 26984 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index 50 to 66, 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) changed from NON_VOTER to VOTER. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } }
I20240502 14:00:02.415144 27252 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.85: config changed from index 56 to 85, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 85 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.428462 27253 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.66: config changed from index 50 to 66, 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) changed from NON_VOTER to VOTER. New config: { opid_index: 66 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } }
I20240502 14:00:02.429850 27254 consensus_queue.cc:237] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 51, Committed index: 51, Last appended: 1.52, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } }
I20240502 14:00:02.433696 26989 raft_consensus.cc:1261] T a734b1d6d928408cac2a9e1b263f2697 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 52. Preceding OpId from leader: term: 1 index: 53. (index mismatch)
I20240502 14:00:02.434271 27383 raft_consensus.cc:1261] T a734b1d6d928408cac2a9e1b263f2697 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 52. Preceding OpId from leader: term: 1 index: 53. (index mismatch)
I20240502 14:00:02.434741 28169 consensus_queue.cc:1035] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 53, Last known committed idx: 51, Time since last communication: 0.000s
I20240502 14:00:02.435041 28531 consensus_queue.cc:1035] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 53, Last known committed idx: 51, Time since last communication: 0.000s
I20240502 14:00:02.436825 28362 raft_consensus.cc:1203] T 41e73be38c4348a5bb86da6c0f6c6d22 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.52->[1.53-1.71] Dedup: 1.63->[1.64-1.71]
I20240502 14:00:02.445989 28707 raft_consensus.cc:1050] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a: attempting to promote NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 to VOTER
I20240502 14:00:02.446461 28707 consensus_queue.cc:237] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 105, Committed index: 105, Last appended: 1.105, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 106 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } }
I20240502 14:00:02.452347 27255 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 66. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
I20240502 14:00:02.457334 27122 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.85: config changed from index 56 to 85, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 85 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.458180 27382 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 105. Preceding OpId from leader: term: 1 index: 106. (index mismatch)
I20240502 14:00:02.458539 27384 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.85: config changed from index 56 to 85, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 85 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.459676 26859 raft_consensus.cc:1261] T a734b1d6d928408cac2a9e1b263f2697 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 52. Preceding OpId from leader: term: 1 index: 53. (index mismatch)
I20240502 14:00:02.460237 26991 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 66. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
I20240502 14:00:02.460652 28365 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 105. Preceding OpId from leader: term: 1 index: 106. (index mismatch)
I20240502 14:00:02.460671 28364 raft_consensus.cc:1261] T a734b1d6d928408cac2a9e1b263f2697 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 52. Preceding OpId from leader: term: 1 index: 53. (index mismatch)
I20240502 14:00:02.461019 28495 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 106, Last known committed idx: 105, Time since last communication: 0.000s
I20240502 14:00:02.461257 28708 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 106, Last known committed idx: 105, Time since last communication: 0.000s
I20240502 14:00:02.465241 26986 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 105. Preceding OpId from leader: term: 1 index: 106. (index mismatch)
I20240502 14:00:02.466663 28495 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 66, Time since last communication: 0.000s
I20240502 14:00:02.467368 27515 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 105. Preceding OpId from leader: term: 1 index: 106. (index mismatch)
I20240502 14:00:02.467782 28708 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 106, Last known committed idx: 105, Time since last communication: 0.000s
I20240502 14:00:02.469439 27251 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 105. Preceding OpId from leader: term: 1 index: 106. (index mismatch)
I20240502 14:00:02.470423 27515 consensus_queue.cc:237] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 101, Committed index: 101, Last appended: 1.101, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 102 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:02.472806 28708 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 106, Last known committed idx: 105, Time since last communication: 0.000s
I20240502 14:00:02.473985 28708 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 66, Time since last communication: 0.000s
I20240502 14:00:02.474256 28708 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 106, Last known committed idx: 105, Time since last communication: 0.000s
I20240502 14:00:02.474510 28169 consensus_queue.cc:1035] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 53, Last known committed idx: 51, Time since last communication: 0.000s
I20240502 14:00:02.476650 28617 consensus_queue.cc:1035] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 53, Last known committed idx: 51, Time since last communication: 0.000s
I20240502 14:00:02.477350 28495 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [term 1 LEADER]: Committing config change with OpId 1.67: config changed from index 66 to 67, VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) evicted. New config: { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } }
I20240502 14:00:02.478042 27383 raft_consensus.cc:1261] T abb00d05d9704f5e8a78cb9fd1866e2a P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 101. Preceding OpId from leader: term: 1 index: 102. (index mismatch)
I20240502 14:00:02.477934 28680 raft_consensus.cc:2932] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 LEADER]: Committing config change with OpId 1.53: config changed from index 49 to 53, VOTER 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70) evicted. New config: { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.478257 28708 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [term 1 LEADER]: Committing config change with OpId 1.106: config changed from index 66 to 106, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 106 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.478506 28860 consensus_queue.cc:1035] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 102, Last known committed idx: 101, Time since last communication: 0.000s
W20240502 14:00:02.478828 27452 consensus_peers.cc:485] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 -> Peer 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71:42201): Couldn't send request to peer 5f9ddfa1f6fc484d8fb96075a8efb649. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: abb00d05d9704f5e8a78cb9fd1866e2a. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:02.479156 27118 raft_consensus.cc:1261] T abb00d05d9704f5e8a78cb9fd1866e2a P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 101. Preceding OpId from leader: term: 1 index: 102. (index mismatch)
I20240502 14:00:02.479651 26859 raft_consensus.cc:1261] T abb00d05d9704f5e8a78cb9fd1866e2a P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 101. Preceding OpId from leader: term: 1 index: 102. (index mismatch)
I20240502 14:00:02.479635 28860 consensus_queue.cc:1035] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 102, Last known committed idx: 100, Time since last communication: 0.000s
I20240502 14:00:02.479697 26991 raft_consensus.cc:1261] T abb00d05d9704f5e8a78cb9fd1866e2a P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 101. Preceding OpId from leader: term: 1 index: 102. (index mismatch)
I20240502 14:00:02.480321 26991 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.67: config changed from index 66 to 67, VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) evicted. New config: { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } }
I20240502 14:00:02.481046 28860 consensus_queue.cc:1035] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 102, Last known committed idx: 100, Time since last communication: 0.000s
I20240502 14:00:02.481253 28615 consensus_queue.cc:1035] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 102, Last known committed idx: 101, Time since last communication: 0.000s
I20240502 14:00:02.482355 28615 raft_consensus.cc:2932] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [term 1 LEADER]: Committing config change with OpId 1.102: config changed from index -1 to 102, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 102 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:02.485471 27383 raft_consensus.cc:2932] T abb00d05d9704f5e8a78cb9fd1866e2a P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.102: config changed from index -1 to 102, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 102 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:02.487571 26987 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.106: config changed from index 66 to 106, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 106 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.488211 26855 raft_consensus.cc:2932] T abb00d05d9704f5e8a78cb9fd1866e2a P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.102: config changed from index -1 to 102, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 102 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:02.490504 27255 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.67: config changed from index 66 to 67, VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) evicted. New config: { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } }
I20240502 14:00:02.490559 26859 raft_consensus.cc:2932] T a734b1d6d928408cac2a9e1b263f2697 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.53: config changed from index 49 to 53, VOTER 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70) evicted. New config: { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.490650 28362 raft_consensus.cc:2932] T a734b1d6d928408cac2a9e1b263f2697 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.53: config changed from index 49 to 53, VOTER 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70) evicted. New config: { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.491065 27251 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.106: config changed from index 66 to 106, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 106 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.494537 28364 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.106: config changed from index 66 to 106, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 106 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.495558 26986 raft_consensus.cc:2932] T a734b1d6d928408cac2a9e1b263f2697 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.53: config changed from index 49 to 53, VOTER 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70) evicted. New config: { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.497951 27517 consensus_queue.cc:237] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 86, Committed index: 86, Last appended: 1.86, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 87 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } }
I20240502 14:00:02.499639 28363 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 86. Preceding OpId from leader: term: 1 index: 87. (index mismatch)
I20240502 14:00:02.499971 28860 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 87, Last known committed idx: 86, Time since last communication: 0.000s
I20240502 14:00:02.501613 26985 raft_consensus.cc:2932] T abb00d05d9704f5e8a78cb9fd1866e2a P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.102: config changed from index -1 to 102, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 102 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:02.505323 27252 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 86. Preceding OpId from leader: term: 1 index: 87. (index mismatch)
I20240502 14:00:02.508296 28860 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 87, Last known committed idx: 86, Time since last communication: 0.000s
I20240502 14:00:02.510372 28860 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [term 1 LEADER]: Committing config change with OpId 1.87: config changed from index 85 to 87, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New config: { opid_index: 87 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.511139 27254 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.87: config changed from index 85 to 87, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New config: { opid_index: 87 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.539291 28864 ts_tablet_manager.cc:1888] T 95f064e56ee241dba95f9a6cd5984d97 P 0280081e4ff74896832ec3985c372b3a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.65
I20240502 14:00:02.539427 28864 log.cc:1196] T 95f064e56ee241dba95f9a6cd5984d97 P 0280081e4ff74896832ec3985c372b3a: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/wals/95f064e56ee241dba95f9a6cd5984d97
I20240502 14:00:02.540196 27387 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.106: config changed from index 66 to 106, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 106 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.542502 27517 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.106: config changed from index 66 to 106, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 106 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.542783 26703 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet a734b1d6d928408cac2a9e1b263f2697 with cas_config_opid_index 49: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:02.542675 27519 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.67: config changed from index 66 to 67, VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) evicted. New config: { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } }
I20240502 14:00:02.542979 26703 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4a74c0c312a7403d89958a3632cab7d3 with cas_config_opid_index 66: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:02.542901 26710 catalog_manager.cc:5484] T a734b1d6d928408cac2a9e1b263f2697 P b4fc3c795b1d4505b3a2903532d38b52 reported cstate change: config changed from index 49 to 53, VOTER 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70) evicted. New cstate: current_term: 1 leader_uuid: "b4fc3c795b1d4505b3a2903532d38b52" committed_config { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:02.543087 26703 catalog_manager.cc:4830] TS 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733): tablet 95f064e56ee241dba95f9a6cd5984d97 (table rebalance_test_table_1 [id=a35cd53b479c486fad6a93e2f4317542]) successfully deleted
I20240502 14:00:02.543303 27385 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 66. Preceding OpId from leader: term: 1 index: 67. (index mismatch)
I20240502 14:00:02.544668 28543 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 67, Last known committed idx: 66, Time since last communication: 0.000s
I20240502 14:00:02.545730 26714 catalog_manager.cc:5484] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a reported cstate change: config changed from index 66 to 106, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0280081e4ff74896832ec3985c372b3a" committed_config { opid_index: 106 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:02.546047 26714 catalog_manager.cc:5484] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a reported cstate change: config changed from index 66 to 67, VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) evicted. New cstate: current_term: 1 leader_uuid: "0280081e4ff74896832ec3985c372b3a" committed_config { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:02.546101 27385 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.67: config changed from index 66 to 67, VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) evicted. New config: { opid_index: 67 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } }
I20240502 14:00:02.546541 27499 tablet_service.cc:1514] Processing DeleteTablet for tablet a734b1d6d928408cac2a9e1b263f2697 with delete_type TABLET_DATA_TOMBSTONED (TS 34b161bd719d4d72baf8da4f40aef136 not found in new config with opid_index 53) from {username='slave'} at 127.0.0.1:33294
I20240502 14:00:02.547106 28875 tablet_replica.cc:330] T a734b1d6d928408cac2a9e1b263f2697 P 34b161bd719d4d72baf8da4f40aef136: stopping tablet replica
I20240502 14:00:02.547619 28875 raft_consensus.cc:2229] T a734b1d6d928408cac2a9e1b263f2697 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:02.547787 28875 pending_rounds.cc:62] T a734b1d6d928408cac2a9e1b263f2697 P 34b161bd719d4d72baf8da4f40aef136: Trying to abort 1 pending ops.
I20240502 14:00:02.547844 28875 pending_rounds.cc:68] T a734b1d6d928408cac2a9e1b263f2697 P 34b161bd719d4d72baf8da4f40aef136: Aborting op as it isn't in flight: id { term: 1 index: 52 } timestamp: 7023240816333099008 op_type: WRITE_OP write_request { tablet_id: "a734b1d6d928408cac2a9e1b263f2697" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000\2028\007Z\371:\035\314\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""1684fcd9e9e86dde" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7023240816026079232 authz_token { token_data: "\010\211\273\316\261\006\"3\n\005slave\022*\n 72ce8466a12e4262b615aa4cc05d3b93\020\001\030\001 \001(\001" signature: "<redacted>""\022\332\023\260\343\343X\367}77i\032`\364\351\227\025\241~`\036\365\243\204mCv\263\365\321\371})\322>\236\357\240\036\031\244\370N\264\356\317\335#E`M|\240\331\021Wh)\261q\242\322\210" signing_key_seq_num: 0 } } request_id { client_id: "83fd83d511794421a3ac4e1d969def01" seq_no: 122 first_incomplete_seq_no: 122 attempt_no: 0 }
I20240502 14:00:02.548226 28875 raft_consensus.cc:2258] T a734b1d6d928408cac2a9e1b263f2697 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:02.548661 26713 catalog_manager.cc:5484] T abb00d05d9704f5e8a78cb9fd1866e2a P d3e36186faa442b7bd87357e74d47f68 reported cstate change: config changed from index -1 to 102, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New cstate: current_term: 1 leader_uuid: "34b161bd719d4d72baf8da4f40aef136" committed_config { opid_index: 102 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:02.549794 27122 consensus_queue.cc:237] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 106, Committed index: 106, Last appended: 1.107, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } }
I20240502 14:00:02.549887 28875 ts_tablet_manager.cc:1875] T a734b1d6d928408cac2a9e1b263f2697 P 34b161bd719d4d72baf8da4f40aef136: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:02.551479 26837 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a74c0c312a7403d89958a3632cab7d3 with delete_type TABLET_DATA_TOMBSTONED (TS d3e36186faa442b7bd87357e74d47f68 not found in new config with opid_index 67) from {username='slave'} at 127.0.0.1:45678
I20240502 14:00:02.552412 27381 raft_consensus.cc:2932] T a734b1d6d928408cac2a9e1b263f2697 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.53: config changed from index 49 to 53, VOTER 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70) evicted. New config: { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.555982 28876 tablet_replica.cc:330] T 4a74c0c312a7403d89958a3632cab7d3 P d3e36186faa442b7bd87357e74d47f68: stopping tablet replica
I20240502 14:00:02.556499 28876 raft_consensus.cc:2229] T 4a74c0c312a7403d89958a3632cab7d3 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:02.556747 28876 raft_consensus.cc:2258] T 4a74c0c312a7403d89958a3632cab7d3 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:02.558005 28876 ts_tablet_manager.cc:1875] T 4a74c0c312a7403d89958a3632cab7d3 P d3e36186faa442b7bd87357e74d47f68: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:02.563436 27123 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 85. Preceding OpId from leader: term: 1 index: 87. (index mismatch)
I20240502 14:00:02.564112 28873 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 87, Last known committed idx: 85, Time since last communication: 0.001s
I20240502 14:00:02.567610 27123 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.87: config changed from index 85 to 87, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New config: { opid_index: 87 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.623493 26984 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 106. Preceding OpId from leader: term: 1 index: 108. (index mismatch)
I20240502 14:00:02.624040 28875 ts_tablet_manager.cc:1888] T a734b1d6d928408cac2a9e1b263f2697 P 34b161bd719d4d72baf8da4f40aef136: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.52
I20240502 14:00:02.624146 28875 log.cc:1196] T a734b1d6d928408cac2a9e1b263f2697 P 34b161bd719d4d72baf8da4f40aef136: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal/wals/a734b1d6d928408cac2a9e1b263f2697
I20240502 14:00:02.625376 26708 catalog_manager.cc:5484] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52 reported cstate change: config changed from index 85 to 87, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New cstate: current_term: 1 leader_uuid: "34b161bd719d4d72baf8da4f40aef136" committed_config { opid_index: 87 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.626360 28362 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 106. Preceding OpId from leader: term: 1 index: 108. (index mismatch)
I20240502 14:00:02.626653 26700 catalog_manager.cc:4830] TS 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217): tablet a734b1d6d928408cac2a9e1b263f2697 (table rebalance_test_table_2 [id=72ce8466a12e4262b615aa4cc05d3b93]) successfully deleted
I20240502 14:00:02.627292 27382 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 86. Preceding OpId from leader: term: 1 index: 87. (index mismatch)
I20240502 14:00:02.628978 26971 tablet_service.cc:1514] Processing DeleteTablet for tablet 9efaa3c68c534606bd37bbe4703b2ab2 with delete_type TABLET_DATA_TOMBSTONED (TS 256ce64303ec49e1a5cd8e0ffee436b0 not found in new config with opid_index 87) from {username='slave'} at 127.0.0.1:51832
I20240502 14:00:02.632273 28495 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 108, Last known committed idx: 106, Time since last communication: 0.000s
I20240502 14:00:02.632579 28707 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 108, Last known committed idx: 106, Time since last communication: 0.000s
I20240502 14:00:02.636276 28128 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 87, Last known committed idx: 85, Time since last communication: 0.000s
I20240502 14:00:02.642273 28449 raft_consensus.cc:1050] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0: attempting to promote NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 to VOTER
I20240502 14:00:02.644403 28449 consensus_queue.cc:237] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 71, Committed index: 71, Last appended: 1.72, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 73 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } }
I20240502 14:00:02.644480 28495 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [term 1 LEADER]: Committing config change with OpId 1.108: config changed from index 106 to 108, VOTER b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68) evicted. New config: { opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.646236 26988 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.108: config changed from index 106 to 108, VOTER b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68) evicted. New config: { opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.646384 27122 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 72. Preceding OpId from leader: term: 1 index: 73. (index mismatch)
I20240502 14:00:02.646302 27382 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.87: config changed from index 85 to 87, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New config: { opid_index: 87 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.646888 28831 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 73, Last known committed idx: 71, Time since last communication: 0.000s
I20240502 14:00:02.647845 28362 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 72. Preceding OpId from leader: term: 1 index: 73. (index mismatch)
I20240502 14:00:02.647966 27384 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 72. Preceding OpId from leader: term: 1 index: 73. (index mismatch)
I20240502 14:00:02.648303 28754 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 73, Last known committed idx: 71, Time since last communication: 0.000s
I20240502 14:00:02.648831 26859 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 72. Preceding OpId from leader: term: 1 index: 73. (index mismatch)
I20240502 14:00:02.648816 28754 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 73, Last known committed idx: 71, Time since last communication: 0.000s
I20240502 14:00:02.649261 28754 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 73, Last known committed idx: 71, Time since last communication: 0.000s
I20240502 14:00:02.651075 28754 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 LEADER]: Committing config change with OpId 1.73: config changed from index 53 to 73, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 73 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.654421 26859 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.73: config changed from index 53 to 73, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 73 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.654868 27251 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 72. Preceding OpId from leader: term: 1 index: 73. (index mismatch)
I20240502 14:00:02.658708 27384 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.73: config changed from index 53 to 73, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 73 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.659157 28019 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 73, Last known committed idx: 71, Time since last communication: 0.000s
I20240502 14:00:02.668115 28877 tablet_replica.cc:330] T 9efaa3c68c534606bd37bbe4703b2ab2 P 256ce64303ec49e1a5cd8e0ffee436b0: stopping tablet replica
I20240502 14:00:02.668551 28877 raft_consensus.cc:2229] T 9efaa3c68c534606bd37bbe4703b2ab2 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:02.668812 27251 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.73: config changed from index 53 to 73, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 73 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.675329 28877 raft_consensus.cc:2258] T 9efaa3c68c534606bd37bbe4703b2ab2 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:02.698794 28877 ts_tablet_manager.cc:1875] T 9efaa3c68c534606bd37bbe4703b2ab2 P 256ce64303ec49e1a5cd8e0ffee436b0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:02.708127 27386 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 106. Preceding OpId from leader: term: 1 index: 108. (index mismatch)
I20240502 14:00:02.708539 26703 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6ed88ddf0d1b4dabbbf4604019705436 with cas_config_opid_index 106: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:02.710758 28708 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 108, Last known committed idx: 106, Time since last communication: 0.000s
I20240502 14:00:02.712642 27381 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.108: config changed from index 106 to 108, VOTER b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68) evicted. New config: { opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.717025 26710 catalog_manager.cc:5484] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a reported cstate change: config changed from index 106 to 108, VOTER b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "0280081e4ff74896832ec3985c372b3a" committed_config { opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:02.718073 26708 catalog_manager.cc:5484] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52 reported cstate change: config changed from index 53 to 73, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" committed_config { opid_index: 73 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.722386 28876 ts_tablet_manager.cc:1888] T 4a74c0c312a7403d89958a3632cab7d3 P d3e36186faa442b7bd87357e74d47f68: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.66
I20240502 14:00:02.722543 28876 log.cc:1196] T 4a74c0c312a7403d89958a3632cab7d3 P d3e36186faa442b7bd87357e74d47f68: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/wals/4a74c0c312a7403d89958a3632cab7d3
I20240502 14:00:02.723345 27234 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ed88ddf0d1b4dabbbf4604019705436 with delete_type TABLET_DATA_TOMBSTONED (TS b4fc3c795b1d4505b3a2903532d38b52 not found in new config with opid_index 108) from {username='slave'} at 127.0.0.1:57834
I20240502 14:00:02.723742 26700 catalog_manager.cc:4830] TS d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841): tablet 4a74c0c312a7403d89958a3632cab7d3 (table rebalance_test_table_1 [id=a35cd53b479c486fad6a93e2f4317542]) successfully deleted
I20240502 14:00:02.727943 28362 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.108: config changed from index 106 to 108, VOTER b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68) evicted. New config: { opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.728420 27122 raft_consensus.cc:2932] T abb00d05d9704f5e8a78cb9fd1866e2a P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.102: config changed from index -1 to 102, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 102 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:02.731261 28882 tablet_replica.cc:330] T 6ed88ddf0d1b4dabbbf4604019705436 P b4fc3c795b1d4505b3a2903532d38b52: stopping tablet replica
I20240502 14:00:02.732152 28882 raft_consensus.cc:2229] T 6ed88ddf0d1b4dabbbf4604019705436 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:02.732748 28882 raft_consensus.cc:2258] T 6ed88ddf0d1b4dabbbf4604019705436 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:02.742205 28882 ts_tablet_manager.cc:1875] T 6ed88ddf0d1b4dabbbf4604019705436 P b4fc3c795b1d4505b3a2903532d38b52: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:02.801678 28877 ts_tablet_manager.cc:1888] T 9efaa3c68c534606bd37bbe4703b2ab2 P 256ce64303ec49e1a5cd8e0ffee436b0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.86
I20240502 14:00:02.801889 28877 log.cc:1196] T 9efaa3c68c534606bd37bbe4703b2ab2 P 256ce64303ec49e1a5cd8e0ffee436b0: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/wals/9efaa3c68c534606bd37bbe4703b2ab2
I20240502 14:00:02.802502 26700 catalog_manager.cc:4830] TS 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473): tablet 9efaa3c68c534606bd37bbe4703b2ab2 (table rebalance_test_table_3 [id=ecda844ae2ee4cb5a179b79c8dacb5ac]) successfully deleted
I20240502 14:00:02.806344 27515 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 106. Preceding OpId from leader: term: 1 index: 108. (index mismatch)
I20240502 14:00:02.813920 28495 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 108, Last known committed idx: 106, Time since last communication: 0.000s
I20240502 14:00:02.815711 27517 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.108: config changed from index 106 to 108, VOTER b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68) evicted. New config: { opid_index: 108 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.881963 28882 ts_tablet_manager.cc:1888] T 6ed88ddf0d1b4dabbbf4604019705436 P b4fc3c795b1d4505b3a2903532d38b52: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.106
I20240502 14:00:02.882172 28882 log.cc:1196] T 6ed88ddf0d1b4dabbbf4604019705436 P b4fc3c795b1d4505b3a2903532d38b52: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/6ed88ddf0d1b4dabbbf4604019705436
I20240502 14:00:02.899374 28364 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.87: config changed from index 85 to 87, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New config: { opid_index: 87 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.899809 28363 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.73: config changed from index 53 to 73, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 73 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.901317 26984 consensus_queue.cc:237] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 73, Committed index: 73, Last appended: 1.74, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 75 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } }
I20240502 14:00:02.902680 27118 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.73: config changed from index 53 to 73, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 73 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.903728 26854 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 75. (index mismatch)
I20240502 14:00:02.905946 28878 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 75, Last known committed idx: 73, Time since last communication: 0.000s
I20240502 14:00:02.909463 26700 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 9efaa3c68c534606bd37bbe4703b2ab2 with cas_config_opid_index 85: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:02.911072 26703 catalog_manager.cc:4830] TS b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68:35167): tablet 6ed88ddf0d1b4dabbbf4604019705436 (table rebalance_test_table_3 [id=ecda844ae2ee4cb5a179b79c8dacb5ac]) successfully deleted
I20240502 14:00:02.913138 26859 raft_consensus.cc:1203] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.73->[1.74-1.74] Dedup: 1.74->[]
I20240502 14:00:02.933615 27253 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 74. Preceding OpId from leader: term: 1 index: 75. (index mismatch)
I20240502 14:00:02.963470 28878 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 75, Last known committed idx: 73, Time since last communication: 0.000s
I20240502 14:00:02.974583 28878 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 LEADER]: Committing config change with OpId 1.75: config changed from index 73 to 75, VOTER e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69) evicted. New config: { opid_index: 75 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:02.975502 27253 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.75: config changed from index 73 to 75, VOTER e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69) evicted. New config: { opid_index: 75 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:03.028685 28916 ts_tablet_manager.cc:902] T abb00d05d9704f5e8a78cb9fd1866e2a P 5f9ddfa1f6fc484d8fb96075a8efb649: Initiating tablet copy from peer 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217)
I20240502 14:00:03.029301 28916 tablet_copy_client.cc:308] T abb00d05d9704f5e8a78cb9fd1866e2a P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.70:38217
I20240502 14:00:03.029817 27540 tablet_copy_service.cc:140] P 34b161bd719d4d72baf8da4f40aef136: Received BeginTabletCopySession request for tablet abb00d05d9704f5e8a78cb9fd1866e2a from peer 5f9ddfa1f6fc484d8fb96075a8efb649 ({username='slave'} at 127.24.66.71:46263)
I20240502 14:00:03.029919 27540 tablet_copy_service.cc:161] P 34b161bd719d4d72baf8da4f40aef136: Beginning new tablet copy session on tablet abb00d05d9704f5e8a78cb9fd1866e2a from peer 5f9ddfa1f6fc484d8fb96075a8efb649 at {username='slave'} at 127.24.66.71:46263: session id = 5f9ddfa1f6fc484d8fb96075a8efb649-abb00d05d9704f5e8a78cb9fd1866e2a
I20240502 14:00:03.030995 27540 tablet_copy_source_session.cc:205] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:03.032560 28916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abb00d05d9704f5e8a78cb9fd1866e2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:03.059686 28916 tablet_copy_client.cc:783] T abb00d05d9704f5e8a78cb9fd1866e2a P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:03.060547 28916 tablet_copy_client.cc:647] T abb00d05d9704f5e8a78cb9fd1866e2a P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:03.060683 27115 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 75. (index mismatch)
I20240502 14:00:03.060906 26710 catalog_manager.cc:5484] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52 reported cstate change: config changed from index 73 to 75, VOTER e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69) evicted. New cstate: current_term: 1 leader_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" committed_config { opid_index: 75 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:03.062724 28365 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 73. Preceding OpId from leader: term: 1 index: 75. (index mismatch)
I20240502 14:00:03.063153 28019 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 75, Last known committed idx: 73, Time since last communication: 0.000s
I20240502 14:00:03.064677 28363 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.75: config changed from index 73 to 75, VOTER e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69) evicted. New config: { opid_index: 75 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:03.067615 28916 tablet_copy_client.cc:523] T abb00d05d9704f5e8a78cb9fd1866e2a P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:03.067795 26700 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 41e73be38c4348a5bb86da6c0f6c6d22 with cas_config_opid_index 73: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:03.071830 28916 tablet_bootstrap.cc:492] T abb00d05d9704f5e8a78cb9fd1866e2a P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap starting.
I20240502 14:00:03.073050 28019 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 75, Last known committed idx: 73, Time since last communication: 0.000s
I20240502 14:00:03.075206 27118 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.75: config changed from index 73 to 75, VOTER e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69) evicted. New config: { opid_index: 75 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:03.076110 26858 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.75: config changed from index 73 to 75, VOTER e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69) evicted. New config: { opid_index: 75 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:03.142796 27517 consensus_queue.cc:237] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 114, Committed index: 114, Last appended: 1.114, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:03.144817 26855 raft_consensus.cc:1261] T fcf8a440cfbe490e9aa735cad599e472 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 114. Preceding OpId from leader: term: 1 index: 115. (index mismatch)
I20240502 14:00:03.145557 28128 consensus_queue.cc:1035] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 115, Last known committed idx: 114, Time since last communication: 0.000s
I20240502 14:00:03.145678 27367 tablet_service.cc:1514] Processing DeleteTablet for tablet 41e73be38c4348a5bb86da6c0f6c6d22 with delete_type TABLET_DATA_TOMBSTONED (TS e44f8c16a14a48a38a5a74af2899c217 not found in new config with opid_index 75) from {username='slave'} at 127.0.0.1:49794
W20240502 14:00:03.145766 27452 consensus_peers.cc:485] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 -> Peer 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71:42201): Couldn't send request to peer 5f9ddfa1f6fc484d8fb96075a8efb649. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fcf8a440cfbe490e9aa735cad599e472. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:03.146852 27117 raft_consensus.cc:1261] T fcf8a440cfbe490e9aa735cad599e472 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 114. Preceding OpId from leader: term: 1 index: 115. (index mismatch)
I20240502 14:00:03.147321 28934 tablet_replica.cc:330] T 41e73be38c4348a5bb86da6c0f6c6d22 P e44f8c16a14a48a38a5a74af2899c217: stopping tablet replica
I20240502 14:00:03.147454 28872 consensus_queue.cc:1035] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 115, Last known committed idx: 114, Time since last communication: 0.000s
I20240502 14:00:03.147822 27253 raft_consensus.cc:1261] T fcf8a440cfbe490e9aa735cad599e472 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 114. Preceding OpId from leader: term: 1 index: 115. (index mismatch)
I20240502 14:00:03.148140 28934 raft_consensus.cc:2229] T 41e73be38c4348a5bb86da6c0f6c6d22 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:03.148294 28872 consensus_queue.cc:1035] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 115, Last known committed idx: 114, Time since last communication: 0.000s
I20240502 14:00:03.150132 28934 raft_consensus.cc:2258] T 41e73be38c4348a5bb86da6c0f6c6d22 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:03.151731 28934 ts_tablet_manager.cc:1875] T 41e73be38c4348a5bb86da6c0f6c6d22 P e44f8c16a14a48a38a5a74af2899c217: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:03.153484 28873 raft_consensus.cc:2932] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [term 1 LEADER]: Committing config change with OpId 1.115: config changed from index -1 to 115, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:03.156751 26713 catalog_manager.cc:5484] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 reported cstate change: config changed from index -1 to 115, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New cstate: current_term: 1 leader_uuid: "34b161bd719d4d72baf8da4f40aef136" committed_config { opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:03.157487 27120 raft_consensus.cc:2932] T fcf8a440cfbe490e9aa735cad599e472 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.115: config changed from index -1 to 115, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:03.160910 26858 raft_consensus.cc:2932] T fcf8a440cfbe490e9aa735cad599e472 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.115: config changed from index -1 to 115, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:03.167084 26988 raft_consensus.cc:1261] T fcf8a440cfbe490e9aa735cad599e472 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 114. Preceding OpId from leader: term: 1 index: 115. (index mismatch)
I20240502 14:00:03.175745 27253 raft_consensus.cc:2932] T fcf8a440cfbe490e9aa735cad599e472 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.115: config changed from index -1 to 115, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:03.185163 28934 ts_tablet_manager.cc:1888] T 41e73be38c4348a5bb86da6c0f6c6d22 P e44f8c16a14a48a38a5a74af2899c217: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.73
I20240502 14:00:03.185283 28934 log.cc:1196] T 41e73be38c4348a5bb86da6c0f6c6d22 P e44f8c16a14a48a38a5a74af2899c217: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal/wals/41e73be38c4348a5bb86da6c0f6c6d22
I20240502 14:00:03.186122 28873 consensus_queue.cc:1035] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 115, Last known committed idx: 114, Time since last communication: 0.000s
I20240502 14:00:03.272506 28916 tablet_bootstrap.cc:492] T abb00d05d9704f5e8a78cb9fd1866e2a P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap replayed 1/1 log segments. Stats: ops{read=110 overwritten=0 applied=110 ignored=0} inserts{seen=782 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:03.272855 28916 tablet_bootstrap.cc:492] T abb00d05d9704f5e8a78cb9fd1866e2a P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap complete.
I20240502 14:00:03.273068 28916 ts_tablet_manager.cc:1367] T abb00d05d9704f5e8a78cb9fd1866e2a P 5f9ddfa1f6fc484d8fb96075a8efb649: Time spent bootstrapping tablet: real 0.201s user 0.098s sys 0.005s
I20240502 14:00:03.273558 28916 raft_consensus.cc:348] T abb00d05d9704f5e8a78cb9fd1866e2a P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 102 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:03.273708 28916 raft_consensus.cc:729] T abb00d05d9704f5e8a78cb9fd1866e2a P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f9ddfa1f6fc484d8fb96075a8efb649, State: Initialized, Role: LEARNER
I20240502 14:00:03.273815 28916 consensus_queue.cc:260] T abb00d05d9704f5e8a78cb9fd1866e2a P 5f9ddfa1f6fc484d8fb96075a8efb649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 110, Last appended: 1.110, Last appended by leader: 110, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 102 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:03.274119 28916 ts_tablet_manager.cc:1398] T abb00d05d9704f5e8a78cb9fd1866e2a P 5f9ddfa1f6fc484d8fb96075a8efb649: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:03.274860 27540 tablet_copy_service.cc:342] P 34b161bd719d4d72baf8da4f40aef136: Request end of tablet copy session 5f9ddfa1f6fc484d8fb96075a8efb649-abb00d05d9704f5e8a78cb9fd1866e2a received from {username='slave'} at 127.24.66.71:46263
I20240502 14:00:03.274932 27540 tablet_copy_service.cc:433] P 34b161bd719d4d72baf8da4f40aef136: ending tablet copy session 5f9ddfa1f6fc484d8fb96075a8efb649-abb00d05d9704f5e8a78cb9fd1866e2a on tablet abb00d05d9704f5e8a78cb9fd1866e2a with peer 5f9ddfa1f6fc484d8fb96075a8efb649
I20240502 14:00:03.288867 26701 catalog_manager.cc:4830] TS e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107): tablet 41e73be38c4348a5bb86da6c0f6c6d22 (table rebalance_test_table_1 [id=a35cd53b479c486fad6a93e2f4317542]) successfully deleted
I20240502 14:00:03.310180 26989 raft_consensus.cc:2932] T fcf8a440cfbe490e9aa735cad599e472 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.115: config changed from index -1 to 115, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:03.493446 27384 consensus_queue.cc:237] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 52, Committed index: 52, Last appended: 1.52, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:03.496156 26855 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 52. Preceding OpId from leader: term: 1 index: 53. (index mismatch)
I20240502 14:00:03.496651 27517 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 52. Preceding OpId from leader: term: 1 index: 53. (index mismatch)
I20240502 14:00:03.497795 28874 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 53, Last known committed idx: 52, Time since last communication: 0.000s
I20240502 14:00:03.498898 28881 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 53, Last known committed idx: 52, Time since last communication: 0.000s
W20240502 14:00:03.499449 27320 consensus_peers.cc:485] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 -> Peer 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71:42201): Couldn't send request to peer 5f9ddfa1f6fc484d8fb96075a8efb649. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 945d054a56ed4043b7b49d14ed917532. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:03.502969 27120 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 52. Preceding OpId from leader: term: 1 index: 53. (index mismatch)
I20240502 14:00:03.504459 28874 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [term 1 LEADER]: Committing config change with OpId 1.53: config changed from index -1 to 53, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:03.508421 26713 catalog_manager.cc:5484] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 reported cstate change: config changed from index -1 to 53, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New cstate: current_term: 1 leader_uuid: "e44f8c16a14a48a38a5a74af2899c217" committed_config { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:03.508527 26856 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.53: config changed from index -1 to 53, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:03.508893 27517 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.53: config changed from index -1 to 53, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:03.511081 28874 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 53, Last known committed idx: 52, Time since last communication: 0.001s
I20240502 14:00:03.517979 27117 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.53: config changed from index -1 to 53, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:03.528949 26988 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 52. Preceding OpId from leader: term: 1 index: 53. (index mismatch)
I20240502 14:00:03.548211 28874 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 53, Last known committed idx: 52, Time since last communication: 0.000s
I20240502 14:00:03.560395 26987 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.53: config changed from index -1 to 53, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:03.567776 28363 raft_consensus.cc:1203] T abb00d05d9704f5e8a78cb9fd1866e2a P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.101->[1.102-1.115] Dedup: 1.110->[1.111-1.115]
I20240502 14:00:03.596858 27251 consensus_queue.cc:237] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 145, Committed index: 145, Last appended: 1.145, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 146 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:03.601266 27115 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 145. Preceding OpId from leader: term: 1 index: 146. (index mismatch)
I20240502 14:00:03.601881 26856 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 144. Preceding OpId from leader: term: 1 index: 146. (index mismatch)
I20240502 14:00:03.602134 27385 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 145. Preceding OpId from leader: term: 1 index: 146. (index mismatch)
I20240502 14:00:03.602844 28531 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 146, Last known committed idx: 145, Time since last communication: 0.000s
I20240502 14:00:03.609882 28941 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 146, Last known committed idx: 145, Time since last communication: 0.000s
W20240502 14:00:03.618533 27188 consensus_peers.cc:485] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 -> Peer 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71:42201): Couldn't send request to peer 5f9ddfa1f6fc484d8fb96075a8efb649. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f4bb31e5c0454e54a7b3855445120783. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:03.620944 28681 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 146, Last known committed idx: 144, Time since last communication: 0.000s
I20240502 14:00:03.621434 28681 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 LEADER]: Committing config change with OpId 1.146: config changed from index -1 to 146, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 146 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:03.622483 26855 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.146: config changed from index -1 to 146, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 146 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:03.625178 26708 catalog_manager.cc:5484] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 reported cstate change: config changed from index -1 to 146, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New cstate: current_term: 1 leader_uuid: "b4fc3c795b1d4505b3a2903532d38b52" committed_config { opid_index: 146 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:03.625375 27117 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.146: config changed from index -1 to 146, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 146 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:03.636271 28916 ts_tablet_manager.cc:902] T fcf8a440cfbe490e9aa735cad599e472 P 5f9ddfa1f6fc484d8fb96075a8efb649: Initiating tablet copy from peer 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217)
I20240502 14:00:03.636826 28916 tablet_copy_client.cc:308] T fcf8a440cfbe490e9aa735cad599e472 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.70:38217
I20240502 14:00:03.637421 27540 tablet_copy_service.cc:140] P 34b161bd719d4d72baf8da4f40aef136: Received BeginTabletCopySession request for tablet fcf8a440cfbe490e9aa735cad599e472 from peer 5f9ddfa1f6fc484d8fb96075a8efb649 ({username='slave'} at 127.24.66.71:46263)
I20240502 14:00:03.637516 27540 tablet_copy_service.cc:161] P 34b161bd719d4d72baf8da4f40aef136: Beginning new tablet copy session on tablet fcf8a440cfbe490e9aa735cad599e472 from peer 5f9ddfa1f6fc484d8fb96075a8efb649 at {username='slave'} at 127.24.66.71:46263: session id = 5f9ddfa1f6fc484d8fb96075a8efb649-fcf8a440cfbe490e9aa735cad599e472
I20240502 14:00:03.638497 27540 tablet_copy_source_session.cc:205] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:03.639576 28916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcf8a440cfbe490e9aa735cad599e472. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:03.633311 27384 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.146: config changed from index -1 to 146, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 146 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:03.641768 28916 tablet_copy_client.cc:783] T fcf8a440cfbe490e9aa735cad599e472 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:03.641911 28916 tablet_copy_client.cc:647] T fcf8a440cfbe490e9aa735cad599e472 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:03.652900 26988 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 144. Preceding OpId from leader: term: 1 index: 146. (index mismatch)
I20240502 14:00:03.679668 28680 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 146, Last known committed idx: 144, Time since last communication: 0.000s
I20240502 14:00:03.686868 28916 tablet_copy_client.cc:523] T fcf8a440cfbe490e9aa735cad599e472 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:03.688417 28916 tablet_bootstrap.cc:492] T fcf8a440cfbe490e9aa735cad599e472 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap starting.
I20240502 14:00:03.716392 28860 raft_consensus.cc:1050] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136: attempting to promote NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 to VOTER
I20240502 14:00:03.716781 28860 consensus_queue.cc:237] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 116, Committed index: 116, Last appended: 1.116, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 117 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } }
I20240502 14:00:03.719410 28362 raft_consensus.cc:1261] T abb00d05d9704f5e8a78cb9fd1866e2a P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 116. Preceding OpId from leader: term: 1 index: 117. (index mismatch)
I20240502 14:00:03.719848 27117 raft_consensus.cc:1261] T abb00d05d9704f5e8a78cb9fd1866e2a P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 116. Preceding OpId from leader: term: 1 index: 117. (index mismatch)
I20240502 14:00:03.719689 26987 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.146: config changed from index -1 to 146, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 146 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:03.720484 28873 consensus_queue.cc:1035] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 117, Last known committed idx: 115, Time since last communication: 0.000s
I20240502 14:00:03.720732 27383 raft_consensus.cc:1261] T abb00d05d9704f5e8a78cb9fd1866e2a P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 115. Preceding OpId from leader: term: 1 index: 117. (index mismatch)
I20240502 14:00:03.721084 28873 consensus_queue.cc:1035] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 117, Last known committed idx: 115, Time since last communication: 0.000s
I20240502 14:00:03.721328 26859 raft_consensus.cc:1261] T abb00d05d9704f5e8a78cb9fd1866e2a P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 116. Preceding OpId from leader: term: 1 index: 117. (index mismatch)
I20240502 14:00:03.722806 27782 consensus_queue.cc:1035] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 117, Last known committed idx: 115, Time since last communication: 0.000s
I20240502 14:00:03.723045 27782 consensus_queue.cc:1035] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 117, Last known committed idx: 116, Time since last communication: 0.000s
I20240502 14:00:03.725876 27384 raft_consensus.cc:1203] T abb00d05d9704f5e8a78cb9fd1866e2a P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.115->[1.116-1.116] Dedup: 1.116->[]
I20240502 14:00:03.726955 28861 raft_consensus.cc:2932] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [term 1 LEADER]: Committing config change with OpId 1.117: config changed from index 102 to 117, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 117 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:03.727458 26859 raft_consensus.cc:2932] T abb00d05d9704f5e8a78cb9fd1866e2a P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.117: config changed from index 102 to 117, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 117 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:03.728178 27117 raft_consensus.cc:2932] T abb00d05d9704f5e8a78cb9fd1866e2a P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.117: config changed from index 102 to 117, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 117 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:03.729167 28362 raft_consensus.cc:2932] T abb00d05d9704f5e8a78cb9fd1866e2a P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.117: config changed from index 102 to 117, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 117 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:03.730759 26710 catalog_manager.cc:5484] T abb00d05d9704f5e8a78cb9fd1866e2a P 0280081e4ff74896832ec3985c372b3a reported cstate change: config changed from index 102 to 117, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34b161bd719d4d72baf8da4f40aef136" committed_config { opid_index: 117 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:03.731586 27385 raft_consensus.cc:2932] T abb00d05d9704f5e8a78cb9fd1866e2a P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.117: config changed from index 102 to 117, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 117 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:03.760792 26986 raft_consensus.cc:1261] T abb00d05d9704f5e8a78cb9fd1866e2a P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 115. Preceding OpId from leader: term: 1 index: 117. (index mismatch)
I20240502 14:00:03.761809 28614 consensus_queue.cc:1035] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 117, Last known committed idx: 115, Time since last communication: 0.000s
I20240502 14:00:03.765264 26989 raft_consensus.cc:1203] T abb00d05d9704f5e8a78cb9fd1866e2a P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.115->[1.116-1.118] Dedup: 1.116->[1.117-1.118]
I20240502 14:00:03.765749 26989 raft_consensus.cc:2932] T abb00d05d9704f5e8a78cb9fd1866e2a P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.117: config changed from index 102 to 117, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 117 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:03.911873 27518 consensus_queue.cc:237] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 119, Committed index: 119, Last appended: 1.119, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 120 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } }
I20240502 14:00:03.914189 28362 raft_consensus.cc:1261] T abb00d05d9704f5e8a78cb9fd1866e2a P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 119. Preceding OpId from leader: term: 1 index: 120. (index mismatch)
I20240502 14:00:03.914710 26855 raft_consensus.cc:1261] T abb00d05d9704f5e8a78cb9fd1866e2a P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 119. Preceding OpId from leader: term: 1 index: 120. (index mismatch)
I20240502 14:00:03.914716 27385 raft_consensus.cc:1261] T abb00d05d9704f5e8a78cb9fd1866e2a P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 119. Preceding OpId from leader: term: 1 index: 120. (index mismatch)
I20240502 14:00:03.915062 28860 consensus_queue.cc:1035] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 120, Last known committed idx: 119, Time since last communication: 0.000s
I20240502 14:00:03.915207 28872 consensus_queue.cc:1035] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 120, Last known committed idx: 119, Time since last communication: 0.000s
I20240502 14:00:03.915318 28873 consensus_queue.cc:1035] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 120, Last known committed idx: 119, Time since last communication: 0.000s
I20240502 14:00:03.917899 28860 raft_consensus.cc:2932] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [term 1 LEADER]: Committing config change with OpId 1.120: config changed from index 117 to 120, VOTER 0280081e4ff74896832ec3985c372b3a (127.24.66.67) evicted. New config: { opid_index: 120 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:03.920871 26700 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet abb00d05d9704f5e8a78cb9fd1866e2a with cas_config_opid_index 117: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:03.921723 27385 raft_consensus.cc:2932] T abb00d05d9704f5e8a78cb9fd1866e2a P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.120: config changed from index 117 to 120, VOTER 0280081e4ff74896832ec3985c372b3a (127.24.66.67) evicted. New config: { opid_index: 120 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:03.921936 26713 catalog_manager.cc:5484] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 reported cstate change: config changed from index 117 to 120, VOTER 0280081e4ff74896832ec3985c372b3a (127.24.66.67) evicted. New cstate: current_term: 1 leader_uuid: "34b161bd719d4d72baf8da4f40aef136" committed_config { opid_index: 120 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:03.923147 28362 raft_consensus.cc:2932] T abb00d05d9704f5e8a78cb9fd1866e2a P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.120: config changed from index 117 to 120, VOTER 0280081e4ff74896832ec3985c372b3a (127.24.66.67) evicted. New config: { opid_index: 120 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:03.924605 27101 tablet_service.cc:1514] Processing DeleteTablet for tablet abb00d05d9704f5e8a78cb9fd1866e2a with delete_type TABLET_DATA_TOMBSTONED (TS 0280081e4ff74896832ec3985c372b3a not found in new config with opid_index 120) from {username='slave'} at 127.0.0.1:51610
I20240502 14:00:03.924695 26855 raft_consensus.cc:2932] T abb00d05d9704f5e8a78cb9fd1866e2a P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.120: config changed from index 117 to 120, VOTER 0280081e4ff74896832ec3985c372b3a (127.24.66.67) evicted. New config: { opid_index: 120 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:03.942566 28977 tablet_replica.cc:330] T abb00d05d9704f5e8a78cb9fd1866e2a P 0280081e4ff74896832ec3985c372b3a: stopping tablet replica
I20240502 14:00:03.942749 28977 raft_consensus.cc:2229] T abb00d05d9704f5e8a78cb9fd1866e2a P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:03.944197 28977 raft_consensus.cc:2258] T abb00d05d9704f5e8a78cb9fd1866e2a P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:03.945626 28977 ts_tablet_manager.cc:1875] T abb00d05d9704f5e8a78cb9fd1866e2a P 0280081e4ff74896832ec3985c372b3a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:03.959745 28977 ts_tablet_manager.cc:1888] T abb00d05d9704f5e8a78cb9fd1866e2a P 0280081e4ff74896832ec3985c372b3a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.119
I20240502 14:00:03.959870 28977 log.cc:1196] T abb00d05d9704f5e8a78cb9fd1866e2a P 0280081e4ff74896832ec3985c372b3a: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/wals/abb00d05d9704f5e8a78cb9fd1866e2a
I20240502 14:00:03.960417 26703 catalog_manager.cc:4830] TS 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733): tablet abb00d05d9704f5e8a78cb9fd1866e2a (table rebalance_test_table_0 [id=360c53214ba44103b4436952fc42eddc]) successfully deleted
I20240502 14:00:03.974213 28916 tablet_bootstrap.cc:492] T fcf8a440cfbe490e9aa735cad599e472 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap replayed 1/1 log segments. Stats: ops{read=118 overwritten=0 applied=118 ignored=0} inserts{seen=803 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:03.974486 28916 tablet_bootstrap.cc:492] T fcf8a440cfbe490e9aa735cad599e472 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap complete.
I20240502 14:00:03.974679 28916 ts_tablet_manager.cc:1367] T fcf8a440cfbe490e9aa735cad599e472 P 5f9ddfa1f6fc484d8fb96075a8efb649: Time spent bootstrapping tablet: real 0.286s user 0.109s sys 0.000s
I20240502 14:00:03.975139 28916 raft_consensus.cc:348] T fcf8a440cfbe490e9aa735cad599e472 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:03.975265 28916 raft_consensus.cc:729] T fcf8a440cfbe490e9aa735cad599e472 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f9ddfa1f6fc484d8fb96075a8efb649, State: Initialized, Role: LEARNER
I20240502 14:00:03.975356 28916 consensus_queue.cc:260] T fcf8a440cfbe490e9aa735cad599e472 P 5f9ddfa1f6fc484d8fb96075a8efb649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 118, Last appended: 1.118, Last appended by leader: 118, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 115 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:03.986074 26985 raft_consensus.cc:1261] T abb00d05d9704f5e8a78cb9fd1866e2a P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 119. Preceding OpId from leader: term: 1 index: 120. (index mismatch)
I20240502 14:00:03.994653 28916 ts_tablet_manager.cc:1398] T fcf8a440cfbe490e9aa735cad599e472 P 5f9ddfa1f6fc484d8fb96075a8efb649: Time spent starting tablet: real 0.020s user 0.003s sys 0.000s
I20240502 14:00:03.995303 27540 tablet_copy_service.cc:342] P 34b161bd719d4d72baf8da4f40aef136: Request end of tablet copy session 5f9ddfa1f6fc484d8fb96075a8efb649-fcf8a440cfbe490e9aa735cad599e472 received from {username='slave'} at 127.24.66.71:46263
I20240502 14:00:03.995379 27540 tablet_copy_service.cc:433] P 34b161bd719d4d72baf8da4f40aef136: ending tablet copy session 5f9ddfa1f6fc484d8fb96075a8efb649-fcf8a440cfbe490e9aa735cad599e472 on tablet fcf8a440cfbe490e9aa735cad599e472 with peer 5f9ddfa1f6fc484d8fb96075a8efb649
I20240502 14:00:03.997668 28916 ts_tablet_manager.cc:902] T 945d054a56ed4043b7b49d14ed917532 P 5f9ddfa1f6fc484d8fb96075a8efb649: Initiating tablet copy from peer e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107)
I20240502 14:00:03.999074 28916 tablet_copy_client.cc:308] T 945d054a56ed4043b7b49d14ed917532 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.69:39107
I20240502 14:00:04.002489 28873 consensus_queue.cc:1035] T abb00d05d9704f5e8a78cb9fd1866e2a P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 120, Last known committed idx: 118, Time since last communication: 0.000s
I20240502 14:00:04.003777 27408 tablet_copy_service.cc:140] P e44f8c16a14a48a38a5a74af2899c217: Received BeginTabletCopySession request for tablet 945d054a56ed4043b7b49d14ed917532 from peer 5f9ddfa1f6fc484d8fb96075a8efb649 ({username='slave'} at 127.24.66.71:49171)
I20240502 14:00:04.003909 27408 tablet_copy_service.cc:161] P e44f8c16a14a48a38a5a74af2899c217: Beginning new tablet copy session on tablet 945d054a56ed4043b7b49d14ed917532 from peer 5f9ddfa1f6fc484d8fb96075a8efb649 at {username='slave'} at 127.24.66.71:49171: session id = 5f9ddfa1f6fc484d8fb96075a8efb649-945d054a56ed4043b7b49d14ed917532
I20240502 14:00:04.005321 27408 tablet_copy_source_session.cc:205] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:04.009790 26987 raft_consensus.cc:2932] T abb00d05d9704f5e8a78cb9fd1866e2a P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.120: config changed from index 117 to 120, VOTER 0280081e4ff74896832ec3985c372b3a (127.24.66.67) evicted. New config: { opid_index: 120 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.011102 27384 consensus_queue.cc:237] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 117, Committed index: 117, Last appended: 1.117, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 118 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:04.011569 28362 raft_consensus.cc:1203] T fcf8a440cfbe490e9aa735cad599e472 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.114->[1.115-1.126] Dedup: 1.118->[1.119-1.126]
I20240502 14:00:04.013299 28916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 945d054a56ed4043b7b49d14ed917532. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:04.017655 28916 tablet_copy_client.cc:783] T 945d054a56ed4043b7b49d14ed917532 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:04.017663 27518 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 117. Preceding OpId from leader: term: 1 index: 118. (index mismatch)
I20240502 14:00:04.017818 28916 tablet_copy_client.cc:647] T 945d054a56ed4043b7b49d14ed917532 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:04.018110 28874 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 118, Last known committed idx: 117, Time since last communication: 0.000s
I20240502 14:00:04.020490 26858 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 117. Preceding OpId from leader: term: 1 index: 118. (index mismatch)
I20240502 14:00:04.021219 26987 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 117. Preceding OpId from leader: term: 1 index: 118. (index mismatch)
W20240502 14:00:04.019981 27320 consensus_peers.cc:485] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 -> Peer 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71:42201): Couldn't send request to peer 5f9ddfa1f6fc484d8fb96075a8efb649. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f7ea57a2ffe04e9e962c61d0e82fc475. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:04.021692 28916 tablet_copy_client.cc:523] T 945d054a56ed4043b7b49d14ed917532 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:04.023001 28874 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 118, Last known committed idx: 117, Time since last communication: 0.000s
I20240502 14:00:04.023734 28874 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 118, Last known committed idx: 117, Time since last communication: 0.000s
I20240502 14:00:04.025298 28916 tablet_bootstrap.cc:492] T 945d054a56ed4043b7b49d14ed917532 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap starting.
I20240502 14:00:04.025578 28228 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [term 1 LEADER]: Committing config change with OpId 1.118: config changed from index -1 to 118, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 118 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:04.028311 26859 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.118: config changed from index -1 to 118, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 118 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:04.032940 26713 catalog_manager.cc:5484] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68 reported cstate change: config changed from index -1 to 118, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New cstate: current_term: 1 leader_uuid: "e44f8c16a14a48a38a5a74af2899c217" committed_config { opid_index: 118 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:04.038692 27120 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 117. Preceding OpId from leader: term: 1 index: 118. (index mismatch)
I20240502 14:00:04.040518 28874 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 118, Last known committed idx: 117, Time since last communication: 0.000s
I20240502 14:00:04.042892 27518 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.118: config changed from index -1 to 118, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 118 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:04.044229 27120 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.118: config changed from index -1 to 118, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 118 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:04.056162 26987 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.118: config changed from index -1 to 118, NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) added. New config: { opid_index: 118 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } }
I20240502 14:00:04.086262 28873 raft_consensus.cc:1050] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136: attempting to promote NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 to VOTER
I20240502 14:00:04.086763 28873 consensus_queue.cc:237] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 127, Committed index: 127, Last appended: 1.127, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 128 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } }
I20240502 14:00:04.088243 27120 raft_consensus.cc:1261] T fcf8a440cfbe490e9aa735cad599e472 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 127. Preceding OpId from leader: term: 1 index: 128. (index mismatch)
I20240502 14:00:04.088758 28861 consensus_queue.cc:1035] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 128, Last known committed idx: 127, Time since last communication: 0.000s
I20240502 14:00:04.089751 26858 raft_consensus.cc:1261] T fcf8a440cfbe490e9aa735cad599e472 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 127. Preceding OpId from leader: term: 1 index: 128. (index mismatch)
I20240502 14:00:04.090708 27782 consensus_queue.cc:1035] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 128, Last known committed idx: 127, Time since last communication: 0.000s
I20240502 14:00:04.092118 27255 raft_consensus.cc:1261] T fcf8a440cfbe490e9aa735cad599e472 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 127. Preceding OpId from leader: term: 1 index: 128. (index mismatch)
I20240502 14:00:04.093050 27782 consensus_queue.cc:1035] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 128, Last known committed idx: 127, Time since last communication: 0.000s
I20240502 14:00:04.094506 28362 raft_consensus.cc:1261] T fcf8a440cfbe490e9aa735cad599e472 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 127. Preceding OpId from leader: term: 1 index: 128. (index mismatch)
I20240502 14:00:04.094902 28861 consensus_queue.cc:1035] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 128, Last known committed idx: 127, Time since last communication: 0.000s
I20240502 14:00:04.094866 27782 raft_consensus.cc:2932] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [term 1 LEADER]: Committing config change with OpId 1.128: config changed from index 115 to 128, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 128 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.095762 28365 raft_consensus.cc:2932] T fcf8a440cfbe490e9aa735cad599e472 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.128: config changed from index 115 to 128, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 128 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.097967 27120 raft_consensus.cc:2932] T fcf8a440cfbe490e9aa735cad599e472 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.128: config changed from index 115 to 128, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 128 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.098342 26708 catalog_manager.cc:5484] T fcf8a440cfbe490e9aa735cad599e472 P 5f9ddfa1f6fc484d8fb96075a8efb649 reported cstate change: config changed from index 115 to 128, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34b161bd719d4d72baf8da4f40aef136" committed_config { opid_index: 128 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.100023 27255 raft_consensus.cc:2932] T fcf8a440cfbe490e9aa735cad599e472 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.128: config changed from index 115 to 128, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 128 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.100836 26858 raft_consensus.cc:2932] T fcf8a440cfbe490e9aa735cad599e472 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.128: config changed from index 115 to 128, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 128 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.111490 26985 raft_consensus.cc:1261] T fcf8a440cfbe490e9aa735cad599e472 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 127. Preceding OpId from leader: term: 1 index: 128. (index mismatch)
I20240502 14:00:04.112717 28873 consensus_queue.cc:1035] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 128, Last known committed idx: 127, Time since last communication: 0.000s
I20240502 14:00:04.114125 26985 raft_consensus.cc:2932] T fcf8a440cfbe490e9aa735cad599e472 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.128: config changed from index 115 to 128, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 128 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.116055 28916 tablet_bootstrap.cc:492] T 945d054a56ed4043b7b49d14ed917532 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap replayed 1/1 log segments. Stats: ops{read=56 overwritten=0 applied=56 ignored=0} inserts{seen=687 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:04.116497 28916 tablet_bootstrap.cc:492] T 945d054a56ed4043b7b49d14ed917532 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap complete.
I20240502 14:00:04.116605 28995 ts_tablet_manager.cc:902] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649: Initiating tablet copy from peer b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68:35167)
I20240502 14:00:04.116694 28916 ts_tablet_manager.cc:1367] T 945d054a56ed4043b7b49d14ed917532 P 5f9ddfa1f6fc484d8fb96075a8efb649: Time spent bootstrapping tablet: real 0.091s user 0.061s sys 0.001s
I20240502 14:00:04.117442 28916 raft_consensus.cc:348] T 945d054a56ed4043b7b49d14ed917532 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:04.117559 28916 raft_consensus.cc:729] T 945d054a56ed4043b7b49d14ed917532 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f9ddfa1f6fc484d8fb96075a8efb649, State: Initialized, Role: LEARNER
I20240502 14:00:04.117664 28916 consensus_queue.cc:260] T 945d054a56ed4043b7b49d14ed917532 P 5f9ddfa1f6fc484d8fb96075a8efb649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 56, Last appended: 1.56, Last appended by leader: 56, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 53 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:04.117714 28995 tablet_copy_client.cc:308] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:35167
I20240502 14:00:04.118105 27275 tablet_copy_service.cc:140] P b4fc3c795b1d4505b3a2903532d38b52: Received BeginTabletCopySession request for tablet f4bb31e5c0454e54a7b3855445120783 from peer 5f9ddfa1f6fc484d8fb96075a8efb649 ({username='slave'} at 127.24.66.71:54201)
I20240502 14:00:04.118208 27275 tablet_copy_service.cc:161] P b4fc3c795b1d4505b3a2903532d38b52: Beginning new tablet copy session on tablet f4bb31e5c0454e54a7b3855445120783 from peer 5f9ddfa1f6fc484d8fb96075a8efb649 at {username='slave'} at 127.24.66.71:54201: session id = 5f9ddfa1f6fc484d8fb96075a8efb649-f4bb31e5c0454e54a7b3855445120783
I20240502 14:00:04.118402 28916 ts_tablet_manager.cc:1398] T 945d054a56ed4043b7b49d14ed917532 P 5f9ddfa1f6fc484d8fb96075a8efb649: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:04.119462 27275 tablet_copy_source_session.cc:205] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:04.119998 27408 tablet_copy_service.cc:342] P e44f8c16a14a48a38a5a74af2899c217: Request end of tablet copy session 5f9ddfa1f6fc484d8fb96075a8efb649-945d054a56ed4043b7b49d14ed917532 received from {username='slave'} at 127.24.66.71:49171
I20240502 14:00:04.120098 27408 tablet_copy_service.cc:433] P e44f8c16a14a48a38a5a74af2899c217: ending tablet copy session 5f9ddfa1f6fc484d8fb96075a8efb649-945d054a56ed4043b7b49d14ed917532 on tablet 945d054a56ed4043b7b49d14ed917532 with peer 5f9ddfa1f6fc484d8fb96075a8efb649
I20240502 14:00:04.122082 28995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4bb31e5c0454e54a7b3855445120783. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:04.124276 28995 tablet_copy_client.cc:783] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:04.125249 28995 tablet_copy_client.cc:647] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:04.159874 27518 consensus_queue.cc:237] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 128, Committed index: 128, Last appended: 1.129, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 130 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } }
I20240502 14:00:04.166788 27117 raft_consensus.cc:1261] T fcf8a440cfbe490e9aa735cad599e472 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 129. Preceding OpId from leader: term: 1 index: 130. (index mismatch)
I20240502 14:00:04.167244 28872 consensus_queue.cc:1035] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 130, Last known committed idx: 128, Time since last communication: 0.000s
I20240502 14:00:04.175504 28365 raft_consensus.cc:1261] T fcf8a440cfbe490e9aa735cad599e472 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 129. Preceding OpId from leader: term: 1 index: 130. (index mismatch)
I20240502 14:00:04.175947 28872 consensus_queue.cc:1035] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 130, Last known committed idx: 128, Time since last communication: 0.000s
I20240502 14:00:04.177397 26988 raft_consensus.cc:1261] T fcf8a440cfbe490e9aa735cad599e472 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 129. Preceding OpId from leader: term: 1 index: 130. (index mismatch)
I20240502 14:00:04.178269 28873 raft_consensus.cc:2932] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [term 1 LEADER]: Committing config change with OpId 1.130: config changed from index 128 to 130, VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) evicted. New config: { opid_index: 130 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.178704 28365 raft_consensus.cc:2932] T fcf8a440cfbe490e9aa735cad599e472 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.130: config changed from index 128 to 130, VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) evicted. New config: { opid_index: 130 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.181485 26700 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet fcf8a440cfbe490e9aa735cad599e472 with cas_config_opid_index 128: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:04.182257 27255 raft_consensus.cc:1261] T fcf8a440cfbe490e9aa735cad599e472 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 129. Preceding OpId from leader: term: 1 index: 130. (index mismatch)
I20240502 14:00:04.182404 26714 catalog_manager.cc:5484] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 reported cstate change: config changed from index 128 to 130, VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) evicted. New cstate: current_term: 1 leader_uuid: "34b161bd719d4d72baf8da4f40aef136" committed_config { opid_index: 130 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:04.185603 27120 raft_consensus.cc:2932] T fcf8a440cfbe490e9aa735cad599e472 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.130: config changed from index 128 to 130, VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) evicted. New config: { opid_index: 130 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.189174 26837 tablet_service.cc:1514] Processing DeleteTablet for tablet fcf8a440cfbe490e9aa735cad599e472 with delete_type TABLET_DATA_TOMBSTONED (TS d3e36186faa442b7bd87357e74d47f68 not found in new config with opid_index 130) from {username='slave'} at 127.0.0.1:45678
I20240502 14:00:04.192310 28969 consensus_queue.cc:1035] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 130, Last known committed idx: 128, Time since last communication: 0.000s
I20240502 14:00:04.193553 29011 tablet_replica.cc:330] T fcf8a440cfbe490e9aa735cad599e472 P d3e36186faa442b7bd87357e74d47f68: stopping tablet replica
I20240502 14:00:04.193740 29011 raft_consensus.cc:2229] T fcf8a440cfbe490e9aa735cad599e472 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:04.194034 29011 pending_rounds.cc:62] T fcf8a440cfbe490e9aa735cad599e472 P d3e36186faa442b7bd87357e74d47f68: Trying to abort 1 pending ops.
I20240502 14:00:04.194162 29011 pending_rounds.cc:68] T fcf8a440cfbe490e9aa735cad599e472 P d3e36186faa442b7bd87357e74d47f68: Aborting op as it isn't in flight: id { term: 1 index: 129 } timestamp: 7023240823416020992 op_type: WRITE_OP write_request { tablet_id: "fcf8a440cfbe490e9aa735cad599e472" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000\033Q\200[8\247\307\276\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""d37491a9940e321a" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7023240823381278720 authz_token { token_data: "\010\211\273\316\261\006\"3\n\005slave\022*\n 56abfd3d24fd434b996a35f2174a0641\020\001\030\001 \001(\001" signature: "<redacted>""\t\307>D\016\rv\211\362kC\'0\301\341\354\253\227\256\212u\234vT\245\307&\256NW~\335<\356{\235Y\216jA\2476\356\301\265(\003\376\274?)]t5\343V4Q\201\224\256\221!\306" signing_key_seq_num: 0 } } request_id { client_id: "4d858e3affbe44cd8a699fdbe3451db4" seq_no: 372 first_incomplete_seq_no: 372 attempt_no: 0 }
I20240502 14:00:04.194715 29011 raft_consensus.cc:2258] T fcf8a440cfbe490e9aa735cad599e472 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:04.195616 29011 ts_tablet_manager.cc:1875] T fcf8a440cfbe490e9aa735cad599e472 P d3e36186faa442b7bd87357e74d47f68: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:04.197943 28995 tablet_copy_client.cc:523] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:04.203233 28995 tablet_bootstrap.cc:492] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap starting.
I20240502 14:00:04.210361 29011 ts_tablet_manager.cc:1888] T fcf8a440cfbe490e9aa735cad599e472 P d3e36186faa442b7bd87357e74d47f68: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.129
I20240502 14:00:04.210539 29011 log.cc:1196] T fcf8a440cfbe490e9aa735cad599e472 P d3e36186faa442b7bd87357e74d47f68: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/wals/fcf8a440cfbe490e9aa735cad599e472
I20240502 14:00:04.210850 26987 raft_consensus.cc:2932] T fcf8a440cfbe490e9aa735cad599e472 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.130: config changed from index 128 to 130, VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) evicted. New config: { opid_index: 130 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.211527 26700 catalog_manager.cc:4830] TS d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841): tablet fcf8a440cfbe490e9aa735cad599e472 (table rebalance_test_table_4 [id=56abfd3d24fd434b996a35f2174a0641]) successfully deleted
I20240502 14:00:04.228744 28861 consensus_queue.cc:1035] T fcf8a440cfbe490e9aa735cad599e472 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 130, Last known committed idx: 128, Time since last communication: 0.000s
I20240502 14:00:04.231027 27253 raft_consensus.cc:2932] T fcf8a440cfbe490e9aa735cad599e472 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.130: config changed from index 128 to 130, VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) evicted. New config: { opid_index: 130 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.428787 28995 tablet_bootstrap.cc:492] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap replayed 1/1 log segments. Stats: ops{read=152 overwritten=0 applied=152 ignored=0} inserts{seen=837 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:04.429136 28995 tablet_bootstrap.cc:492] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap complete.
I20240502 14:00:04.429330 28995 ts_tablet_manager.cc:1367] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649: Time spent bootstrapping tablet: real 0.226s user 0.107s sys 0.017s
I20240502 14:00:04.429773 28995 raft_consensus.cc:348] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 146 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:04.429904 28995 raft_consensus.cc:729] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f9ddfa1f6fc484d8fb96075a8efb649, State: Initialized, Role: LEARNER
I20240502 14:00:04.429997 28995 consensus_queue.cc:260] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 152, Last appended: 1.152, Last appended by leader: 152, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 146 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:04.430243 28995 ts_tablet_manager.cc:1398] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:04.430688 27275 tablet_copy_service.cc:342] P b4fc3c795b1d4505b3a2903532d38b52: Request end of tablet copy session 5f9ddfa1f6fc484d8fb96075a8efb649-f4bb31e5c0454e54a7b3855445120783 received from {username='slave'} at 127.24.66.71:54201
I20240502 14:00:04.430768 27275 tablet_copy_service.cc:433] P b4fc3c795b1d4505b3a2903532d38b52: ending tablet copy session 5f9ddfa1f6fc484d8fb96075a8efb649-f4bb31e5c0454e54a7b3855445120783 on tablet f4bb31e5c0454e54a7b3855445120783 with peer 5f9ddfa1f6fc484d8fb96075a8efb649
I20240502 14:00:04.483404 28365 raft_consensus.cc:1203] T 945d054a56ed4043b7b49d14ed917532 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.52->[1.53-1.58] Dedup: 1.56->[1.57-1.58]
I20240502 14:00:04.538303 28995 ts_tablet_manager.cc:902] T f7ea57a2ffe04e9e962c61d0e82fc475 P 5f9ddfa1f6fc484d8fb96075a8efb649: Initiating tablet copy from peer e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107)
I20240502 14:00:04.538743 28995 tablet_copy_client.cc:308] T f7ea57a2ffe04e9e962c61d0e82fc475 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.69:39107
I20240502 14:00:04.539173 27408 tablet_copy_service.cc:140] P e44f8c16a14a48a38a5a74af2899c217: Received BeginTabletCopySession request for tablet f7ea57a2ffe04e9e962c61d0e82fc475 from peer 5f9ddfa1f6fc484d8fb96075a8efb649 ({username='slave'} at 127.24.66.71:49171)
I20240502 14:00:04.539287 27408 tablet_copy_service.cc:161] P e44f8c16a14a48a38a5a74af2899c217: Beginning new tablet copy session on tablet f7ea57a2ffe04e9e962c61d0e82fc475 from peer 5f9ddfa1f6fc484d8fb96075a8efb649 at {username='slave'} at 127.24.66.71:49171: session id = 5f9ddfa1f6fc484d8fb96075a8efb649-f7ea57a2ffe04e9e962c61d0e82fc475
I20240502 14:00:04.540481 27408 tablet_copy_source_session.cc:205] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:04.542246 28995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7ea57a2ffe04e9e962c61d0e82fc475. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:04.545270 28995 tablet_copy_client.cc:783] T f7ea57a2ffe04e9e962c61d0e82fc475 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:04.545681 28995 tablet_copy_client.cc:647] T f7ea57a2ffe04e9e962c61d0e82fc475 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:04.561829 28995 tablet_copy_client.cc:523] T f7ea57a2ffe04e9e962c61d0e82fc475 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:04.563231 28995 tablet_bootstrap.cc:492] T f7ea57a2ffe04e9e962c61d0e82fc475 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap starting.
I20240502 14:00:04.659754 28551 raft_consensus.cc:1050] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217: attempting to promote NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 to VOTER
I20240502 14:00:04.660281 28551 consensus_queue.cc:237] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 59, Committed index: 59, Last appended: 1.59, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } }
I20240502 14:00:04.662545 27518 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 59. Preceding OpId from leader: term: 1 index: 60. (index mismatch)
I20240502 14:00:04.663818 28228 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 60, Last known committed idx: 59, Time since last communication: 0.000s
I20240502 14:00:04.663887 26858 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 59. Preceding OpId from leader: term: 1 index: 60. (index mismatch)
I20240502 14:00:04.663892 26987 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 59. Preceding OpId from leader: term: 1 index: 60. (index mismatch)
I20240502 14:00:04.664572 28228 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 60, Last known committed idx: 59, Time since last communication: 0.000s
I20240502 14:00:04.664688 28365 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 59. Preceding OpId from leader: term: 1 index: 60. (index mismatch)
I20240502 14:00:04.664863 28228 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 60, Last known committed idx: 59, Time since last communication: 0.000s
I20240502 14:00:04.665745 28228 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 60, Last known committed idx: 59, Time since last communication: 0.000s
I20240502 14:00:04.669291 28881 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [term 1 LEADER]: Committing config change with OpId 1.60: config changed from index 53 to 60, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.671031 28362 raft_consensus.cc:1203] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.145->[1.146-1.163] Dedup: 1.152->[1.153-1.163]
I20240502 14:00:04.672127 26987 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.60: config changed from index 53 to 60, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.674554 26713 catalog_manager.cc:5484] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 reported cstate change: config changed from index 53 to 60, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e44f8c16a14a48a38a5a74af2899c217" committed_config { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:04.677243 26858 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.60: config changed from index 53 to 60, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.679049 28365 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.60: config changed from index 53 to 60, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.677788 27518 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.60: config changed from index 53 to 60, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.699875 27117 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 59. Preceding OpId from leader: term: 1 index: 60. (index mismatch)
I20240502 14:00:04.717397 28881 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 60, Last known committed idx: 58, Time since last communication: 0.000s
I20240502 14:00:04.719605 27120 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.60: config changed from index 53 to 60, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 60 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.753330 28832 raft_consensus.cc:1050] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52: attempting to promote NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 to VOTER
I20240502 14:00:04.753726 28832 consensus_queue.cc:237] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 165, Committed index: 165, Last appended: 1.166, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 167 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } }
I20240502 14:00:04.755182 27117 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 166. Preceding OpId from leader: term: 1 index: 167. (index mismatch)
I20240502 14:00:04.755229 27385 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 166. Preceding OpId from leader: term: 1 index: 167. (index mismatch)
I20240502 14:00:04.755817 28942 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 167, Last known committed idx: 165, Time since last communication: 0.000s
I20240502 14:00:04.756024 28941 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 167, Last known committed idx: 165, Time since last communication: 0.000s
I20240502 14:00:04.761073 26857 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 164. Preceding OpId from leader: term: 1 index: 167. (index mismatch)
I20240502 14:00:04.761848 26987 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 165. Preceding OpId from leader: term: 1 index: 167. (index mismatch)
I20240502 14:00:04.762038 28365 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 166. Preceding OpId from leader: term: 1 index: 167. (index mismatch)
I20240502 14:00:04.765808 28941 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 167, Last known committed idx: 164, Time since last communication: 0.000s
I20240502 14:00:04.770627 28681 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 167, Last known committed idx: 165, Time since last communication: 0.000s
I20240502 14:00:04.770886 28681 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 167, Last known committed idx: 164, Time since last communication: 0.000s
I20240502 14:00:04.774571 28717 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 LEADER]: Committing config change with OpId 1.167: config changed from index 146 to 167, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 167 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.778100 26714 catalog_manager.cc:5484] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 reported cstate change: config changed from index 146 to 167, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b4fc3c795b1d4505b3a2903532d38b52" committed_config { opid_index: 167 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:04.781064 27385 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.167: config changed from index 146 to 167, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 167 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.781275 26987 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.167: config changed from index 146 to 167, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 167 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.782008 27117 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.167: config changed from index 146 to 167, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 167 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.789784 27253 consensus_queue.cc:237] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 167, Committed index: 167, Last appended: 1.167, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 168 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } }
I20240502 14:00:04.791399 27117 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 167. Preceding OpId from leader: term: 1 index: 168. (index mismatch)
I20240502 14:00:04.791822 28942 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 168, Last known committed idx: 167, Time since last communication: 0.000s
I20240502 14:00:04.792558 27384 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 167. Preceding OpId from leader: term: 1 index: 168. (index mismatch)
I20240502 14:00:04.794200 28941 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 168, Last known committed idx: 167, Time since last communication: 0.000s
I20240502 14:00:04.794749 26985 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 167. Preceding OpId from leader: term: 1 index: 168. (index mismatch)
I20240502 14:00:04.794893 28363 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 167. Preceding OpId from leader: term: 1 index: 168. (index mismatch)
I20240502 14:00:04.795094 28365 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.167: config changed from index 146 to 167, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 167 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.802131 28941 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 168, Last known committed idx: 167, Time since last communication: 0.000s
I20240502 14:00:04.802002 27385 consensus_queue.cc:237] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 61, Committed index: 61, Last appended: 1.61, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } }
I20240502 14:00:04.806058 28941 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 168, Last known committed idx: 165, Time since last communication: 0.000s
I20240502 14:00:04.806702 28941 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 LEADER]: Committing config change with OpId 1.168: config changed from index 167 to 168, VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) evicted. New config: { opid_index: 168 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.807659 26987 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.168: config changed from index 167 to 168, VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) evicted. New config: { opid_index: 168 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.809398 26703 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f4bb31e5c0454e54a7b3855445120783 with cas_config_opid_index 167: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:04.810556 27519 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 61. Preceding OpId from leader: term: 1 index: 62. (index mismatch)
I20240502 14:00:04.810781 26713 catalog_manager.cc:5484] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 reported cstate change: config changed from index 167 to 168, VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) evicted. New cstate: current_term: 1 leader_uuid: "b4fc3c795b1d4505b3a2903532d38b52" committed_config { opid_index: 168 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:04.811101 26859 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 61. Preceding OpId from leader: term: 1 index: 62. (index mismatch)
I20240502 14:00:04.812034 28228 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 62, Last known committed idx: 61, Time since last communication: 0.000s
I20240502 14:00:04.812285 28228 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 62, Last known committed idx: 61, Time since last communication: 0.000s
I20240502 14:00:04.814116 26837 tablet_service.cc:1514] Processing DeleteTablet for tablet f4bb31e5c0454e54a7b3855445120783 with delete_type TABLET_DATA_TOMBSTONED (TS d3e36186faa442b7bd87357e74d47f68 not found in new config with opid_index 168) from {username='slave'} at 127.0.0.1:45678
I20240502 14:00:04.818419 28363 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.168: config changed from index 167 to 168, VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) evicted. New config: { opid_index: 168 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.819718 28881 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [term 1 LEADER]: Committing config change with OpId 1.62: config changed from index 60 to 62, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New config: { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.823232 26859 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.62: config changed from index 60 to 62, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New config: { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.825647 26714 catalog_manager.cc:5484] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68 reported cstate change: config changed from index 60 to 62, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New cstate: current_term: 1 leader_uuid: "e44f8c16a14a48a38a5a74af2899c217" committed_config { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.826395 27385 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.168: config changed from index 167 to 168, VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) evicted. New config: { opid_index: 168 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.826730 26701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 945d054a56ed4043b7b49d14ed917532 with cas_config_opid_index 60: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:04.827261 27518 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.62: config changed from index 60 to 62, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New config: { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.829123 28362 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 61. Preceding OpId from leader: term: 1 index: 62. (index mismatch)
I20240502 14:00:04.829645 28881 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 62, Last known committed idx: 60, Time since last communication: 0.000s
I20240502 14:00:04.830169 26971 tablet_service.cc:1514] Processing DeleteTablet for tablet 945d054a56ed4043b7b49d14ed917532 with delete_type TABLET_DATA_TOMBSTONED (TS 256ce64303ec49e1a5cd8e0ffee436b0 not found in new config with opid_index 62) from {username='slave'} at 127.0.0.1:51832
I20240502 14:00:04.831576 28363 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.62: config changed from index 60 to 62, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New config: { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.831404 27118 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.168: config changed from index 167 to 168, VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) evicted. New config: { opid_index: 168 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:04.842840 29058 tablet_replica.cc:330] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68: stopping tablet replica
I20240502 14:00:04.843088 29059 tablet_replica.cc:330] T 945d054a56ed4043b7b49d14ed917532 P 256ce64303ec49e1a5cd8e0ffee436b0: stopping tablet replica
I20240502 14:00:04.843286 29059 raft_consensus.cc:2229] T 945d054a56ed4043b7b49d14ed917532 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:04.843484 29058 raft_consensus.cc:2229] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:04.844266 29058 pending_rounds.cc:62] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68: Trying to abort 2 pending ops.
I20240502 14:00:04.844456 29059 raft_consensus.cc:2258] T 945d054a56ed4043b7b49d14ed917532 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:04.846719 29059 ts_tablet_manager.cc:1875] T 945d054a56ed4043b7b49d14ed917532 P 256ce64303ec49e1a5cd8e0ffee436b0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:04.846843 29058 pending_rounds.cc:68] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68: Aborting op as it isn't in flight: id { term: 1 index: 167 } timestamp: 7023240825870319616 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "f4bb31e5c0454e54a7b3855445120783" old_config { opid_index: 146 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } } } new_config { opid_index: 167 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } } }
I20240502 14:00:04.847343 29058 raft_consensus.cc:2894] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Aborting config change with OpId 1.167: Aborted: Op aborted
I20240502 14:00:04.847482 29058 raft_consensus.cc:2866] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20240502 14:00:04.847569 29058 pending_rounds.cc:68] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68: Aborting op as it isn't in flight: id { term: 1 index: 166 } timestamp: 7023240825858031616 op_type: WRITE_OP write_request { tablet_id: "f4bb31e5c0454e54a7b3855445120783" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000/1\203.&2\016\026\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""24b240f7658e55e6" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7023240825772896256 authz_token { token_data: "\010\211\273\316\261\006\"3\n\005slave\022*\n 360c53214ba44103b4436952fc42eddc\020\001\030\001 \001(\001" signature: "<redacted>""\253<\315\237m\312sb&\210-\252\214\r\217\271b\025\374#B\362p\277\304V\234]\206\225/\347\237J\341zIyv\307\274-wh\320\306\226+\"k?\177VP\260\330\316\013\240l(P\351|" signing_key_seq_num: 0 } } request_id { client_id: "0ded900c1a8f40a0a941140d50439912" seq_no: 411 first_incomplete_seq_no: 411 attempt_no: 0 }
I20240502 14:00:04.848852 29058 raft_consensus.cc:2258] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Raft consensus is shut down!
W20240502 14:00:04.846943 27452 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.66.66:40473, user_credentials={real_user=slave}} blocked reactor thread for 69377.3us
I20240502 14:00:04.852430 29058 ts_tablet_manager.cc:1875] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:04.861353 29059 ts_tablet_manager.cc:1888] T 945d054a56ed4043b7b49d14ed917532 P 256ce64303ec49e1a5cd8e0ffee436b0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.61
I20240502 14:00:04.861590 29059 log.cc:1196] T 945d054a56ed4043b7b49d14ed917532 P 256ce64303ec49e1a5cd8e0ffee436b0: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/wals/945d054a56ed4043b7b49d14ed917532
I20240502 14:00:04.862211 26700 catalog_manager.cc:4830] TS 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473): tablet 945d054a56ed4043b7b49d14ed917532 (table rebalance_test_table_2 [id=72ce8466a12e4262b615aa4cc05d3b93]) successfully deleted
I20240502 14:00:04.863690 29058 ts_tablet_manager.cc:1888] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.167
I20240502 14:00:04.864858 29058 log.cc:1196] T f4bb31e5c0454e54a7b3855445120783 P d3e36186faa442b7bd87357e74d47f68: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/wals/f4bb31e5c0454e54a7b3855445120783
I20240502 14:00:04.868932 26700 catalog_manager.cc:4830] TS d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841): tablet f4bb31e5c0454e54a7b3855445120783 (table rebalance_test_table_0 [id=360c53214ba44103b4436952fc42eddc]) successfully deleted
I20240502 14:00:04.869817 27120 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 61. Preceding OpId from leader: term: 1 index: 62. (index mismatch)
I20240502 14:00:04.899849 29057 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 62, Last known committed idx: 60, Time since last communication: 0.000s
I20240502 14:00:04.956561 27120 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.62: config changed from index 60 to 62, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New config: { opid_index: 62 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:05.117559 28995 tablet_bootstrap.cc:492] T f7ea57a2ffe04e9e962c61d0e82fc475 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap replayed 1/1 log segments. Stats: ops{read=127 overwritten=0 applied=127 ignored=0} inserts{seen=758 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:05.117973 28995 tablet_bootstrap.cc:492] T f7ea57a2ffe04e9e962c61d0e82fc475 P 5f9ddfa1f6fc484d8fb96075a8efb649: Bootstrap complete.
I20240502 14:00:05.118278 28995 ts_tablet_manager.cc:1367] T f7ea57a2ffe04e9e962c61d0e82fc475 P 5f9ddfa1f6fc484d8fb96075a8efb649: Time spent bootstrapping tablet: real 0.555s user 0.104s sys 0.009s
I20240502 14:00:05.118798 28995 raft_consensus.cc:348] T f7ea57a2ffe04e9e962c61d0e82fc475 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 118 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:05.119076 28995 raft_consensus.cc:729] T f7ea57a2ffe04e9e962c61d0e82fc475 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f9ddfa1f6fc484d8fb96075a8efb649, State: Initialized, Role: LEARNER
I20240502 14:00:05.119228 28995 consensus_queue.cc:260] T f7ea57a2ffe04e9e962c61d0e82fc475 P 5f9ddfa1f6fc484d8fb96075a8efb649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 127, Last appended: 1.127, Last appended by leader: 127, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 118 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: true } }
I20240502 14:00:05.119598 28995 ts_tablet_manager.cc:1398] T f7ea57a2ffe04e9e962c61d0e82fc475 P 5f9ddfa1f6fc484d8fb96075a8efb649: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:05.120005 27408 tablet_copy_service.cc:342] P e44f8c16a14a48a38a5a74af2899c217: Request end of tablet copy session 5f9ddfa1f6fc484d8fb96075a8efb649-f7ea57a2ffe04e9e962c61d0e82fc475 received from {username='slave'} at 127.24.66.71:49171
I20240502 14:00:05.120082 27408 tablet_copy_service.cc:433] P e44f8c16a14a48a38a5a74af2899c217: ending tablet copy session 5f9ddfa1f6fc484d8fb96075a8efb649-f7ea57a2ffe04e9e962c61d0e82fc475 on tablet f7ea57a2ffe04e9e962c61d0e82fc475 with peer 5f9ddfa1f6fc484d8fb96075a8efb649
W20240502 14:00:05.125061 27452 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.66.71:42201, user_credentials={real_user=slave}} blocked reactor thread for 48711us
I20240502 14:00:05.401433 28365 raft_consensus.cc:1203] T f7ea57a2ffe04e9e962c61d0e82fc475 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.117->[1.118-1.136] Dedup: 1.127->[1.128-1.136]
I20240502 14:00:05.441262 28874 raft_consensus.cc:1050] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217: attempting to promote NON_VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 to VOTER
I20240502 14:00:05.441735 28874 consensus_queue.cc:237] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 137, Committed index: 137, Last appended: 1.137, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 138 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } }
I20240502 14:00:05.449472 26856 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 137. Preceding OpId from leader: term: 1 index: 138. (index mismatch)
I20240502 14:00:05.450212 27515 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 137. Preceding OpId from leader: term: 1 index: 138. (index mismatch)
I20240502 14:00:05.450853 28362 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 LEARNER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 137. Preceding OpId from leader: term: 1 index: 138. (index mismatch)
I20240502 14:00:05.455952 29055 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 138, Last known committed idx: 137, Time since last communication: 0.000s
I20240502 14:00:05.456296 29055 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 138, Last known committed idx: 137, Time since last communication: 0.000s
I20240502 14:00:05.456593 29055 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 138, Last known committed idx: 137, Time since last communication: 0.000s
I20240502 14:00:05.458850 26988 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 137. Preceding OpId from leader: term: 1 index: 138. (index mismatch)
I20240502 14:00:05.461114 27115 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 137. Preceding OpId from leader: term: 1 index: 138. (index mismatch)
I20240502 14:00:05.465135 28228 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [term 1 LEADER]: Committing config change with OpId 1.138: config changed from index 118 to 138, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 138 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:05.467171 27515 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.138: config changed from index 118 to 138, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 138 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:05.467449 28365 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.138: config changed from index 118 to 138, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 138 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:05.468174 26710 catalog_manager.cc:5484] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 reported cstate change: config changed from index 118 to 138, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e44f8c16a14a48a38a5a74af2899c217" committed_config { opid_index: 138 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:05.469781 28874 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 138, Last known committed idx: 137, Time since last communication: 0.000s
I20240502 14:00:05.471778 26856 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.138: config changed from index 118 to 138, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 138 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:05.472883 27115 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.138: config changed from index 118 to 138, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 138 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:05.491782 28874 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 138, Last known committed idx: 136, Time since last communication: 0.000s
I20240502 14:00:05.504789 26989 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.138: config changed from index 118 to 138, 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 138 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
W20240502 14:00:05.550454 28228 consensus_queue.cc:1159] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" responder_term: 1 status { last_received { term: 1 index: 140 } last_committed_idx: 139 last_received_current_leader { term: 1 index: 140 } }
I20240502 14:00:05.550659 27384 consensus_queue.cc:237] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 141, Committed index: 141, Last appended: 1.141, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 142 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } }
I20240502 14:00:05.555547 27515 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 141. Preceding OpId from leader: term: 1 index: 142. (index mismatch)
I20240502 14:00:05.555627 26856 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 141. Preceding OpId from leader: term: 1 index: 142. (index mismatch)
I20240502 14:00:05.556208 28363 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 141. Preceding OpId from leader: term: 1 index: 142. (index mismatch)
I20240502 14:00:05.556586 29055 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 142, Last known committed idx: 141, Time since last communication: 0.000s
I20240502 14:00:05.556715 28228 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 142, Last known committed idx: 141, Time since last communication: 0.000s
I20240502 14:00:05.561002 29055 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [term 1 LEADER]: Committing config change with OpId 1.142: config changed from index 138 to 142, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New config: { opid_index: 142 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:05.564112 27120 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 141. Preceding OpId from leader: term: 1 index: 142. (index mismatch)
I20240502 14:00:05.564074 26713 catalog_manager.cc:5484] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 reported cstate change: config changed from index 138 to 142, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New cstate: current_term: 1 leader_uuid: "e44f8c16a14a48a38a5a74af2899c217" committed_config { opid_index: 142 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:05.569391 28874 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 142, Last known committed idx: 141, Time since last communication: 0.000s
I20240502 14:00:05.569600 26701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f7ea57a2ffe04e9e962c61d0e82fc475 with cas_config_opid_index 138: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:05.570452 28881 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 142, Last known committed idx: 141, Time since last communication: 0.000s
I20240502 14:00:05.571260 26858 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.142: config changed from index 138 to 142, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New config: { opid_index: 142 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:05.576284 28363 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.142: config changed from index 138 to 142, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New config: { opid_index: 142 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:05.577466 27515 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.142: config changed from index 138 to 142, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New config: { opid_index: 142 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:05.578361 26971 tablet_service.cc:1514] Processing DeleteTablet for tablet f7ea57a2ffe04e9e962c61d0e82fc475 with delete_type TABLET_DATA_TOMBSTONED (TS 256ce64303ec49e1a5cd8e0ffee436b0 not found in new config with opid_index 142) from {username='slave'} at 127.0.0.1:51832
I20240502 14:00:05.578763 29096 tablet_replica.cc:330] T f7ea57a2ffe04e9e962c61d0e82fc475 P 256ce64303ec49e1a5cd8e0ffee436b0: stopping tablet replica
I20240502 14:00:05.579432 27115 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.142: config changed from index 138 to 142, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New config: { opid_index: 142 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } }
I20240502 14:00:05.580072 29096 raft_consensus.cc:2229] T f7ea57a2ffe04e9e962c61d0e82fc475 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:05.580448 29096 pending_rounds.cc:62] T f7ea57a2ffe04e9e962c61d0e82fc475 P 256ce64303ec49e1a5cd8e0ffee436b0: Trying to abort 1 pending ops.
I20240502 14:00:05.581310 29096 pending_rounds.cc:68] T f7ea57a2ffe04e9e962c61d0e82fc475 P 256ce64303ec49e1a5cd8e0ffee436b0: Aborting op as it isn't in flight: id { term: 1 index: 140 } timestamp: 7023240828966326272 op_type: WRITE_OP write_request { tablet_id: "f7ea57a2ffe04e9e962c61d0e82fc475" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000d\240=Q\241R\242\317\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000" indirect_data: "<redacted>""75cbeaf93d5f3a0d" } external_consistency_mode: CLIENT_PROPAGATED propagated_timestamp: 7023240828936761344 authz_token { token_data: "\010\211\273\316\261\006\"3\n\005slave\022*\n 56abfd3d24fd434b996a35f2174a0641\020\001\030\001 \001(\001" signature: "<redacted>""\t\307>D\016\rv\211\362kC\'0\301\341\354\253\227\256\212u\234vT\245\307&\256NW~\335<\356{\235Y\216jA\2476\356\301\265(\003\376\274?)]t5\343V4Q\201\224\256\221!\306" signing_key_seq_num: 0 } } request_id { client_id: "4d858e3affbe44cd8a699fdbe3451db4" seq_no: 426 first_incomplete_seq_no: 426 attempt_no: 0 }
I20240502 14:00:05.581851 29096 raft_consensus.cc:2258] T f7ea57a2ffe04e9e962c61d0e82fc475 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:05.582760 29096 ts_tablet_manager.cc:1875] T f7ea57a2ffe04e9e962c61d0e82fc475 P 256ce64303ec49e1a5cd8e0ffee436b0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:05.588570 29096 ts_tablet_manager.cc:1888] T f7ea57a2ffe04e9e962c61d0e82fc475 P 256ce64303ec49e1a5cd8e0ffee436b0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.140
I20240502 14:00:05.588733 29096 log.cc:1196] T f7ea57a2ffe04e9e962c61d0e82fc475 P 256ce64303ec49e1a5cd8e0ffee436b0: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/wals/f7ea57a2ffe04e9e962c61d0e82fc475
I20240502 14:00:05.589509 26700 catalog_manager.cc:4830] TS 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473): tablet f7ea57a2ffe04e9e962c61d0e82fc475 (table rebalance_test_table_4 [id=56abfd3d24fd434b996a35f2174a0641]) successfully deleted
W20240502 14:00:05.865900 27188 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.66.71:42201, user_credentials={real_user=slave}} blocked reactor thread for 78771.6us
W20240502 14:00:05.938146 27452 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.66.71:42201, user_credentials={real_user=slave}} blocked reactor thread for 44327.8us
I20240502 14:00:06.197000 27499 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:00:06.229627 26971 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:06.232390 27101 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:06.235559 28345 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:06.236279 26837 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:06.247822 27367 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:00:06.320124 27234 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:00:06.687902 27115 consensus_queue.cc:237] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 186, Committed index: 186, Last appended: 1.187, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 188 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: true } }
I20240502 14:00:06.690738 27381 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 186. Preceding OpId from leader: term: 1 index: 188. (index mismatch)
I20240502 14:00:06.691094 29087 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 188, Last known committed idx: 186, Time since last communication: 0.000s
I20240502 14:00:06.697291 28362 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 186. Preceding OpId from leader: term: 1 index: 188. (index mismatch)
I20240502 14:00:06.698261 26984 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 186. Preceding OpId from leader: term: 1 index: 188. (index mismatch)
I20240502 14:00:06.699005 27518 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 186. Preceding OpId from leader: term: 1 index: 188. (index mismatch)
I20240502 14:00:06.706576 28707 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 188, Last known committed idx: 186, Time since last communication: 0.000s
I20240502 14:00:06.706854 28707 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 188, Last known committed idx: 185, Time since last communication: 0.000s
I20240502 14:00:06.707715 28707 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 188, Last known committed idx: 186, Time since last communication: 0.000s
W20240502 14:00:06.708166 27056 consensus_peers.cc:485] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a -> Peer d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841): Couldn't send request to peer d3e36186faa442b7bd87357e74d47f68. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6ed88ddf0d1b4dabbbf4604019705436. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:06.722301 29088 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [term 1 LEADER]: Committing config change with OpId 1.188: config changed from index 108 to 188, NON_VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) added. New config: { opid_index: 188 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: true } } }
I20240502 14:00:06.727278 26713 catalog_manager.cc:5484] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a reported cstate change: config changed from index 108 to 188, NON_VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) added. New cstate: current_term: 1 leader_uuid: "0280081e4ff74896832ec3985c372b3a" committed_config { opid_index: 188 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:06.730870 27381 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.188: config changed from index 108 to 188, NON_VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) added. New config: { opid_index: 188 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: true } } }
I20240502 14:00:06.740859 27518 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.188: config changed from index 108 to 188, NON_VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) added. New config: { opid_index: 188 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: true } } }
I20240502 14:00:06.746300 28362 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.188: config changed from index 108 to 188, NON_VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) added. New config: { opid_index: 188 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: true } } }
I20240502 14:00:06.751601 26985 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.188: config changed from index 108 to 188, NON_VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) added. New config: { opid_index: 188 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: true } } }
W20240502 14:00:07.032229 27452 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.66.71:42201, user_credentials={real_user=slave}} blocked reactor thread for 52168.7us
I20240502 14:00:07.214244 29172 ts_tablet_manager.cc:902] T 6ed88ddf0d1b4dabbbf4604019705436 P d3e36186faa442b7bd87357e74d47f68: Initiating tablet copy from peer 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733)
I20240502 14:00:07.221253 29172 tablet_copy_client.cc:308] T 6ed88ddf0d1b4dabbbf4604019705436 P d3e36186faa442b7bd87357e74d47f68: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.67:35733
I20240502 14:00:07.233757 27144 tablet_copy_service.cc:140] P 0280081e4ff74896832ec3985c372b3a: Received BeginTabletCopySession request for tablet 6ed88ddf0d1b4dabbbf4604019705436 from peer d3e36186faa442b7bd87357e74d47f68 ({username='slave'} at 127.24.66.65:43053)
I20240502 14:00:07.233863 27144 tablet_copy_service.cc:161] P 0280081e4ff74896832ec3985c372b3a: Beginning new tablet copy session on tablet 6ed88ddf0d1b4dabbbf4604019705436 from peer d3e36186faa442b7bd87357e74d47f68 at {username='slave'} at 127.24.66.65:43053: session id = d3e36186faa442b7bd87357e74d47f68-6ed88ddf0d1b4dabbbf4604019705436
I20240502 14:00:07.234872 27144 tablet_copy_source_session.cc:205] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:07.251120 29172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ed88ddf0d1b4dabbbf4604019705436. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:07.343842 29172 tablet_copy_client.cc:783] T 6ed88ddf0d1b4dabbbf4604019705436 P d3e36186faa442b7bd87357e74d47f68: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:07.344201 29172 tablet_copy_client.cc:647] T 6ed88ddf0d1b4dabbbf4604019705436 P d3e36186faa442b7bd87357e74d47f68: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:07.414718 29172 tablet_copy_client.cc:523] T 6ed88ddf0d1b4dabbbf4604019705436 P d3e36186faa442b7bd87357e74d47f68: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:07.421775 29172 tablet_bootstrap.cc:492] T 6ed88ddf0d1b4dabbbf4604019705436 P d3e36186faa442b7bd87357e74d47f68: Bootstrap starting.
I20240502 14:00:08.170646 29172 tablet_bootstrap.cc:492] T 6ed88ddf0d1b4dabbbf4604019705436 P d3e36186faa442b7bd87357e74d47f68: Bootstrap replayed 1/1 log segments. Stats: ops{read=201 overwritten=0 applied=200 ignored=0} inserts{seen=828 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 14:00:08.171228 29172 tablet_bootstrap.cc:492] T 6ed88ddf0d1b4dabbbf4604019705436 P d3e36186faa442b7bd87357e74d47f68: Bootstrap complete.
I20240502 14:00:08.171545 29172 ts_tablet_manager.cc:1367] T 6ed88ddf0d1b4dabbbf4604019705436 P d3e36186faa442b7bd87357e74d47f68: Time spent bootstrapping tablet: real 0.750s user 0.156s sys 0.013s
I20240502 14:00:08.172084 29172 raft_consensus.cc:348] T 6ed88ddf0d1b4dabbbf4604019705436 P d3e36186faa442b7bd87357e74d47f68 [term 1 LEARNER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: 188 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: true } }
I20240502 14:00:08.172814 29172 raft_consensus.cc:729] T 6ed88ddf0d1b4dabbbf4604019705436 P d3e36186faa442b7bd87357e74d47f68 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d3e36186faa442b7bd87357e74d47f68, State: Initialized, Role: LEARNER
I20240502 14:00:08.172994 29172 consensus_queue.cc:260] T 6ed88ddf0d1b4dabbbf4604019705436 P d3e36186faa442b7bd87357e74d47f68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 200, Last appended: 1.201, Last appended by leader: 201, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 188 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: true } }
I20240502 14:00:08.175191 29172 ts_tablet_manager.cc:1398] T 6ed88ddf0d1b4dabbbf4604019705436 P d3e36186faa442b7bd87357e74d47f68: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:00:08.177196 27144 tablet_copy_service.cc:342] P 0280081e4ff74896832ec3985c372b3a: Request end of tablet copy session d3e36186faa442b7bd87357e74d47f68-6ed88ddf0d1b4dabbbf4604019705436 received from {username='slave'} at 127.24.66.65:43053
I20240502 14:00:08.177278 27144 tablet_copy_service.cc:433] P 0280081e4ff74896832ec3985c372b3a: ending tablet copy session d3e36186faa442b7bd87357e74d47f68-6ed88ddf0d1b4dabbbf4604019705436 on tablet 6ed88ddf0d1b4dabbbf4604019705436 with peer d3e36186faa442b7bd87357e74d47f68
I20240502 14:00:08.234994 26858 raft_consensus.cc:1203] T 6ed88ddf0d1b4dabbbf4604019705436 P d3e36186faa442b7bd87357e74d47f68 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.187->[1.188-1.226] Dedup: 1.201->[1.202-1.226]
I20240502 14:00:08.272118 28543 raft_consensus.cc:1050] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a: attempting to promote NON_VOTER d3e36186faa442b7bd87357e74d47f68 to VOTER
I20240502 14:00:08.277992 28543 consensus_queue.cc:237] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 228, Committed index: 228, Last appended: 1.228, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 229 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: false } }
I20240502 14:00:08.279958 27383 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 228. Preceding OpId from leader: term: 1 index: 229. (index mismatch)
I20240502 14:00:08.281122 29088 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 229, Last known committed idx: 228, Time since last communication: 0.000s
I20240502 14:00:08.291522 26856 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P d3e36186faa442b7bd87357e74d47f68 [term 1 LEARNER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 228. Preceding OpId from leader: term: 1 index: 229. (index mismatch)
I20240502 14:00:08.293927 28363 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 228. Preceding OpId from leader: term: 1 index: 229. (index mismatch)
I20240502 14:00:08.295086 27515 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 228. Preceding OpId from leader: term: 1 index: 229. (index mismatch)
I20240502 14:00:08.295835 29200 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 229, Last known committed idx: 228, Time since last communication: 0.000s
I20240502 14:00:08.296303 29200 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 229, Last known committed idx: 228, Time since last communication: 0.000s
I20240502 14:00:08.296824 29200 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 229, Last known committed idx: 228, Time since last communication: 0.000s
I20240502 14:00:08.301831 29088 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [term 1 LEADER]: Committing config change with OpId 1.229: config changed from index 188 to 229, d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) changed from NON_VOTER to VOTER. New config: { opid_index: 229 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: false } } }
I20240502 14:00:08.306120 26714 catalog_manager.cc:5484] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a reported cstate change: config changed from index 188 to 229, d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0280081e4ff74896832ec3985c372b3a" committed_config { opid_index: 229 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:08.307643 27383 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.229: config changed from index 188 to 229, d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) changed from NON_VOTER to VOTER. New config: { opid_index: 229 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: false } } }
I20240502 14:00:08.309409 26988 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 228. Preceding OpId from leader: term: 1 index: 229. (index mismatch)
I20240502 14:00:08.311901 27515 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.229: config changed from index 188 to 229, d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) changed from NON_VOTER to VOTER. New config: { opid_index: 229 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: false } } }
I20240502 14:00:08.317310 28365 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.229: config changed from index 188 to 229, d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) changed from NON_VOTER to VOTER. New config: { opid_index: 229 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: false } } }
I20240502 14:00:08.323056 29200 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 229, Last known committed idx: 228, Time since last communication: 0.000s
I20240502 14:00:08.325078 26856 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.229: config changed from index 188 to 229, d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) changed from NON_VOTER to VOTER. New config: { opid_index: 229 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: false } } }
I20240502 14:00:08.327838 26987 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.229: config changed from index 188 to 229, d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) changed from NON_VOTER to VOTER. New config: { opid_index: 229 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: false } } }
I20240502 14:00:08.581274 27117 consensus_queue.cc:237] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 235, Committed index: 235, Last appended: 1.235, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 236 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: false } }
I20240502 14:00:08.583122 26855 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 235. Preceding OpId from leader: term: 1 index: 236. (index mismatch)
I20240502 14:00:08.585274 26984 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 235. Preceding OpId from leader: term: 1 index: 236. (index mismatch)
I20240502 14:00:08.587334 27519 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 235. Preceding OpId from leader: term: 1 index: 236. (index mismatch)
I20240502 14:00:08.588016 28543 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 236, Last known committed idx: 234, Time since last communication: 0.000s
I20240502 14:00:08.590107 28543 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 236, Last known committed idx: 235, Time since last communication: 0.000s
I20240502 14:00:08.590353 28543 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 236, Last known committed idx: 235, Time since last communication: 0.000s
I20240502 14:00:08.595954 28363 raft_consensus.cc:1261] T 6ed88ddf0d1b4dabbbf4604019705436 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 235. Preceding OpId from leader: term: 1 index: 236. (index mismatch)
I20240502 14:00:08.603847 28707 consensus_queue.cc:1035] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 236, Last known committed idx: 235, Time since last communication: 0.000s
I20240502 14:00:08.604209 28707 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a [term 1 LEADER]: Committing config change with OpId 1.236: config changed from index 229 to 236, VOTER e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69) evicted. New config: { opid_index: 236 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: false } } }
I20240502 14:00:08.607208 26703 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6ed88ddf0d1b4dabbbf4604019705436 with cas_config_opid_index 229: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:08.608532 26710 catalog_manager.cc:5484] T 6ed88ddf0d1b4dabbbf4604019705436 P 0280081e4ff74896832ec3985c372b3a reported cstate change: config changed from index 229 to 236, VOTER e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69) evicted. New cstate: current_term: 1 leader_uuid: "0280081e4ff74896832ec3985c372b3a" committed_config { opid_index: 236 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20240502 14:00:08.637938 29212 log.cc:924] Time spent T 00000000000000000000000000000000 P 60bad1ef960c4e09b6fb0fe11be17da2: Append to log took a long time: real 0.068s user 0.000s sys 0.000s
I20240502 14:00:08.641036 26988 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.236: config changed from index 229 to 236, VOTER e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69) evicted. New config: { opid_index: 236 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: false } } }
I20240502 14:00:08.654516 28361 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.236: config changed from index 229 to 236, VOTER e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69) evicted. New config: { opid_index: 236 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: false } } }
I20240502 14:00:08.654924 27519 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.236: config changed from index 229 to 236, VOTER e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69) evicted. New config: { opid_index: 236 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: false } } }
I20240502 14:00:08.656530 27367 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ed88ddf0d1b4dabbbf4604019705436 with delete_type TABLET_DATA_TOMBSTONED (TS e44f8c16a14a48a38a5a74af2899c217 not found in new config with opid_index 236) from {username='slave'} at 127.0.0.1:49794
I20240502 14:00:08.658872 26856 raft_consensus.cc:2932] T 6ed88ddf0d1b4dabbbf4604019705436 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.236: config changed from index 229 to 236, VOTER e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69) evicted. New config: { opid_index: 236 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { promote: false } } }
I20240502 14:00:08.817698 29234 tablet_replica.cc:330] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217: stopping tablet replica
I20240502 14:00:08.822316 29234 raft_consensus.cc:2229] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:08.831984 29234 raft_consensus.cc:2258] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:08.840870 29234 ts_tablet_manager.cc:1875] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:08.877665 29234 ts_tablet_manager.cc:1888] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.235
W20240502 14:00:08.883953 27452 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.66.71:42201, user_credentials={real_user=slave}} blocked reactor thread for 55837us
I20240502 14:00:08.912524 29234 log.cc:1196] T 6ed88ddf0d1b4dabbbf4604019705436 P e44f8c16a14a48a38a5a74af2899c217: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal/wals/6ed88ddf0d1b4dabbbf4604019705436
I20240502 14:00:08.921128 26701 catalog_manager.cc:4830] TS e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107): tablet 6ed88ddf0d1b4dabbbf4604019705436 (table rebalance_test_table_3 [id=ecda844ae2ee4cb5a179b79c8dacb5ac]) successfully deleted
I20240502 14:00:09.984514 27101 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:10.040499 26971 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:10.070454 28345 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20240502 14:00:10.107611 27323 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.66.68:35167, user_credentials={real_user=slave}} blocked reactor thread for 67183.3us
I20240502 14:00:10.177901 26837 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:10.214684 27499 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:00:10.237689 27367 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:00:10.316613 27234 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
W20240502 14:00:10.463613 27188 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.66.65:40841, user_credentials={real_user=slave}} blocked reactor thread for 64402.8us
W20240502 14:00:10.660192 27188 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.66.71:42201, user_credentials={real_user=slave}} blocked reactor thread for 61507.8us
I20240502 14:00:10.700749 27499 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:00:10.721117 27234 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:00:10.731140 28345 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20240502 14:00:10.752741 27059 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.66.68:35167, user_credentials={real_user=slave}} blocked reactor thread for 48289.1us
I20240502 14:00:10.755824 27101 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:10.772087 27367 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:00:10.812829 26837 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:10.817219 26971 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
W20240502 14:00:12.110769 27056 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.66.65:40841, user_credentials={real_user=slave}} blocked reactor thread for 55829us
I20240502 14:00:12.351850 26971 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:12.359086 27234 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:00:12.371759 28345 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:12.375635 26837 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:12.441823 27101 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:12.443691 27367 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:00:12.482488 27499 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:00:13.164954 27499 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:00:13.167999 28345 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:13.189833 26837 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:13.200516 27234 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:00:13.323781 27101 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:13.374938 26971 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:13.399278 27367 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:00:13.668763 26971 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:13.670159 27499 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:00:13.672897 27101 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:13.681186 28345 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:13.692454 27234 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:00:13.701740 27367 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:00:13.706264 26837 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
60bad1ef960c4e09b6fb0fe11be17da2 | 127.24.66.126:45215 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
allow_unsafe_replication_factor | true | runtime,unsafe | all 1 server(s) checked
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.24.66.84:37497 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
0280081e4ff74896832ec3985c372b3a | 127.24.66.67:35733 | HEALTHY | <none> | 2 | 0
256ce64303ec49e1a5cd8e0ffee436b0 | 127.24.66.66:40473 | HEALTHY | <none> | 1 | 0
34b161bd719d4d72baf8da4f40aef136 | 127.24.66.70:38217 | HEALTHY | <none> | 3 | 0
5f9ddfa1f6fc484d8fb96075a8efb649 | 127.24.66.71:42201 | HEALTHY | <none> | 0 | 0
b4fc3c795b1d4505b3a2903532d38b52 | 127.24.66.68:35167 | HEALTHY | <none> | 5 | 0
d3e36186faa442b7bd87357e74d47f68 | 127.24.66.65:40841 | HEALTHY | <none> | 1 | 0
e44f8c16a14a48a38a5a74af2899c217 | 127.24.66.69:39107 | HEALTHY | <none> | 4 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 7
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 7 server(s) checked
local_ip_for_outbound_sockets | 127.24.66.65 | experimental | 127.24.66.65:40841
local_ip_for_outbound_sockets | 127.24.66.66 | experimental | 127.24.66.66:40473
local_ip_for_outbound_sockets | 127.24.66.67 | experimental | 127.24.66.67:35733
local_ip_for_outbound_sockets | 127.24.66.68 | experimental | 127.24.66.68:35167
local_ip_for_outbound_sockets | 127.24.66.69 | experimental | 127.24.66.69:39107
local_ip_for_outbound_sockets | 127.24.66.70 | experimental | 127.24.66.70:38217
local_ip_for_outbound_sockets | 127.24.66.71 | experimental | 127.24.66.71:42201
never_fsync | true | unsafe,advanced | all 7 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 7 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 7 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 7 server(s) checked
server_dump_info_format | pb | hidden | all 7 server(s) checked
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.24.66.65:40841
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.24.66.66:40473
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.24.66.67:35733
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.24.66.68:35167
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.24.66.69:39107
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.24.66.70:38217
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.24.66.71:42201
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.24.66.84:37497 | all 7 server(s) checked
time_source | builtin | all 7 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 8 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 5 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 11
First Quartile | 11
Median | 11
Third Quartile | 12
Maximum | 12
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 7
Tables | 6
Tablets | 16
Replicas | 80
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240502 14:00:13.733310 24841 log_verifier.cc:126] Checking tablet 1ad8b348f02043dfa4736276dc306895
I20240502 14:00:13.909868 24841 log_verifier.cc:177] Verified matching terms for 350 ops in tablet 1ad8b348f02043dfa4736276dc306895
I20240502 14:00:13.910106 24841 log_verifier.cc:126] Checking tablet 41e73be38c4348a5bb86da6c0f6c6d22
I20240502 14:00:14.038587 24841 log_verifier.cc:177] Verified matching terms for 222 ops in tablet 41e73be38c4348a5bb86da6c0f6c6d22
I20240502 14:00:14.038762 24841 log_verifier.cc:126] Checking tablet 4a156115301a4d40bfc7f2bd11f135bf
I20240502 14:00:14.074235 24841 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 4a156115301a4d40bfc7f2bd11f135bf
I20240502 14:00:14.074301 24841 log_verifier.cc:126] Checking tablet 4a74c0c312a7403d89958a3632cab7d3
I20240502 14:00:14.200949 24841 log_verifier.cc:177] Verified matching terms for 208 ops in tablet 4a74c0c312a7403d89958a3632cab7d3
I20240502 14:00:14.201102 24841 log_verifier.cc:126] Checking tablet 53a68b75cbce41d69432f21f2cfbf256
I20240502 14:00:14.341672 24841 log_verifier.cc:177] Verified matching terms for 292 ops in tablet 53a68b75cbce41d69432f21f2cfbf256
I20240502 14:00:14.341907 24841 log_verifier.cc:126] Checking tablet 6ed88ddf0d1b4dabbbf4604019705436
I20240502 14:00:14.502578 24841 log_verifier.cc:177] Verified matching terms for 352 ops in tablet 6ed88ddf0d1b4dabbbf4604019705436
I20240502 14:00:14.502822 24841 log_verifier.cc:126] Checking tablet 945d054a56ed4043b7b49d14ed917532
I20240502 14:00:14.604761 24841 log_verifier.cc:177] Verified matching terms for 151 ops in tablet 945d054a56ed4043b7b49d14ed917532
I20240502 14:00:14.604928 24841 log_verifier.cc:126] Checking tablet 95f064e56ee241dba95f9a6cd5984d97
I20240502 14:00:14.716387 24841 log_verifier.cc:177] Verified matching terms for 196 ops in tablet 95f064e56ee241dba95f9a6cd5984d97
I20240502 14:00:14.716567 24841 log_verifier.cc:126] Checking tablet 9efaa3c68c534606bd37bbe4703b2ab2
I20240502 14:00:14.853811 24841 log_verifier.cc:177] Verified matching terms for 288 ops in tablet 9efaa3c68c534606bd37bbe4703b2ab2
I20240502 14:00:14.854043 24841 log_verifier.cc:126] Checking tablet a734b1d6d928408cac2a9e1b263f2697
I20240502 14:00:14.961740 24841 log_verifier.cc:177] Verified matching terms for 188 ops in tablet a734b1d6d928408cac2a9e1b263f2697
I20240502 14:00:14.961876 24841 log_verifier.cc:126] Checking tablet abb00d05d9704f5e8a78cb9fd1866e2a
I20240502 14:00:15.107792 24841 log_verifier.cc:177] Verified matching terms for 307 ops in tablet abb00d05d9704f5e8a78cb9fd1866e2a
I20240502 14:00:15.107987 24841 log_verifier.cc:126] Checking tablet d6fcbe92870d40a6b4a860f5f4f81c29
I20240502 14:00:15.236948 24841 log_verifier.cc:177] Verified matching terms for 267 ops in tablet d6fcbe92870d40a6b4a860f5f4f81c29
I20240502 14:00:15.237133 24841 log_verifier.cc:126] Checking tablet df55bedff83e4a5d822547e0fa1c2f87
I20240502 14:00:15.337019 24841 log_verifier.cc:177] Verified matching terms for 178 ops in tablet df55bedff83e4a5d822547e0fa1c2f87
I20240502 14:00:15.337165 24841 log_verifier.cc:126] Checking tablet f4bb31e5c0454e54a7b3855445120783
I20240502 14:00:15.482331 24841 log_verifier.cc:177] Verified matching terms for 318 ops in tablet f4bb31e5c0454e54a7b3855445120783
I20240502 14:00:15.482528 24841 log_verifier.cc:126] Checking tablet f7ea57a2ffe04e9e962c61d0e82fc475
I20240502 14:00:15.618144 24841 log_verifier.cc:177] Verified matching terms for 262 ops in tablet f7ea57a2ffe04e9e962c61d0e82fc475
I20240502 14:00:15.618355 24841 log_verifier.cc:126] Checking tablet fcf8a440cfbe490e9aa735cad599e472
I20240502 14:00:15.742223 24841 log_verifier.cc:177] Verified matching terms for 243 ops in tablet fcf8a440cfbe490e9aa735cad599e472
I20240502 14:00:15.744096 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.72:0
--local_ip_for_outbound_sockets=127.24.66.72
--webserver_interface=127.24.66.72
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45215
--builtin_ntp_servers=127.24.66.84:37497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 14:00:15.864993 29526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:15.865212 29526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:15.865288 29526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:15.868574 29526 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:15.868654 29526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:15.868731 29526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.72
I20240502 14:00:15.872655 29526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:37497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.72:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.24.66.72
--webserver_port=0
--tserver_master_addrs=127.24.66.126:45215
--never_fsync=true
--heap_profile_path=/tmp/kudu.29526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.72
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:15.873553 29526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:15.874647 29526 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:15.883284 29534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:15.883893 29532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:15.884797 29531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:15.885613 29526 server_base.cc:1034] running on GCE node
I20240502 14:00:15.885903 29526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:15.886348 29526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:15.887487 29526 hybrid_clock.cc:648] HybridClock initialized: now 1714658415887458 us; error 39 us; skew 500 ppm
I20240502 14:00:15.889173 29526 webserver.cc:460] Webserver started at http://127.24.66.72:37771/ using document root <none> and password file <none>
I20240502 14:00:15.889683 29526 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:15.889736 29526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:15.889910 29526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:15.891602 29526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "b15496b8de1b4bc084cdfbb7c581f361"
format_stamp: "Formatted at 2024-05-02 14:00:15 on dist-test-slave-5npg"
I20240502 14:00:15.892112 29526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "b15496b8de1b4bc084cdfbb7c581f361"
format_stamp: "Formatted at 2024-05-02 14:00:15 on dist-test-slave-5npg"
I20240502 14:00:15.895723 29526 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20240502 14:00:15.898041 29540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:15.899089 29526 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:15.899232 29526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "b15496b8de1b4bc084cdfbb7c581f361"
format_stamp: "Formatted at 2024-05-02 14:00:15 on dist-test-slave-5npg"
I20240502 14:00:15.899320 29526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:15.935470 29526 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:15.936198 29526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:15.936388 29526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:15.936982 29526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:15.938030 29526 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:15.938088 29526 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:15.938151 29526 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:15.938179 29526 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:15.947911 29526 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.72:46715
I20240502 14:00:15.947937 29654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.72:46715 every 8 connection(s)
I20240502 14:00:15.948881 29526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 14:00:15.951196 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 29526
I20240502 14:00:15.951323 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240502 14:00:15.957815 29655 heartbeater.cc:347] Connected to a master server at 127.24.66.126:45215
I20240502 14:00:15.958005 29655 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:15.958580 29655 heartbeater.cc:511] Master 127.24.66.126:45215 requested a full tablet report, sending...
I20240502 14:00:15.959543 26714 ts_manager.cc:194] Registered new tserver with Master: b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72:46715)
I20240502 14:00:15.960206 26714 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.72:47077
I20240502 14:00:16.146677 26971 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:16.147207 27101 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:16.148386 29588 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:16.157091 27499 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:00:16.157596 28345 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:16.166824 27367 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:00:16.178791 26837 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:16.186350 27234 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:00:16.276149 26971 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:16.276824 27367 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:00:16.277400 27499 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:00:16.279563 28345 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:16.280861 27101 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:16.292192 26837 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:16.293450 29588 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:16.308387 27234 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:00:16.348552 27254 consensus_queue.cc:237] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 350, Committed index: 350, Last appended: 1.350, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 351 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:16.350231 27117 raft_consensus.cc:1261] T 1ad8b348f02043dfa4736276dc306895 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 350. Preceding OpId from leader: term: 1 index: 351. (index mismatch)
I20240502 14:00:16.350711 27381 raft_consensus.cc:1261] T 1ad8b348f02043dfa4736276dc306895 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 350. Preceding OpId from leader: term: 1 index: 351. (index mismatch)
I20240502 14:00:16.350819 26856 raft_consensus.cc:1261] T 1ad8b348f02043dfa4736276dc306895 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 350. Preceding OpId from leader: term: 1 index: 351. (index mismatch)
I20240502 14:00:16.350819 26985 raft_consensus.cc:1261] T 1ad8b348f02043dfa4736276dc306895 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 350. Preceding OpId from leader: term: 1 index: 351. (index mismatch)
I20240502 14:00:16.351253 29750 consensus_queue.cc:1035] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 351, Last known committed idx: 350, Time since last communication: 0.000s
I20240502 14:00:16.351500 29523 consensus_queue.cc:1035] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 351, Last known committed idx: 350, Time since last communication: 0.000s
I20240502 14:00:16.351660 29751 consensus_queue.cc:1035] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 351, Last known committed idx: 350, Time since last communication: 0.000s
I20240502 14:00:16.351891 29752 consensus_queue.cc:1035] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 351, Last known committed idx: 350, Time since last communication: 0.000s
I20240502 14:00:16.353891 29750 raft_consensus.cc:2932] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 LEADER]: Committing config change with OpId 1.351: config changed from index -1 to 351, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 351 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:16.354596 27381 raft_consensus.cc:2932] T 1ad8b348f02043dfa4736276dc306895 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.351: config changed from index -1 to 351, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 351 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:16.354660 27117 raft_consensus.cc:2932] T 1ad8b348f02043dfa4736276dc306895 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.351: config changed from index -1 to 351, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 351 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:16.357496 26714 catalog_manager.cc:5484] T 1ad8b348f02043dfa4736276dc306895 P e44f8c16a14a48a38a5a74af2899c217 reported cstate change: config changed from index -1 to 351, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "b4fc3c795b1d4505b3a2903532d38b52" committed_config { opid_index: 351 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:16.357558 26985 raft_consensus.cc:2932] T 1ad8b348f02043dfa4736276dc306895 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.351: config changed from index -1 to 351, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 351 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
W20240502 14:00:16.359385 27191 consensus_peers.cc:485] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 -> Peer b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72:46715): Couldn't send request to peer b15496b8de1b4bc084cdfbb7c581f361. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1ad8b348f02043dfa4736276dc306895. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:16.361107 26856 raft_consensus.cc:2932] T 1ad8b348f02043dfa4736276dc306895 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.351: config changed from index -1 to 351, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 351 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:16.368824 26985 consensus_queue.cc:237] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 222, Committed index: 222, Last appended: 1.222, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 223 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:16.370450 27254 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 222. Preceding OpId from leader: term: 1 index: 223. (index mismatch)
I20240502 14:00:16.370705 28365 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 222. Preceding OpId from leader: term: 1 index: 223. (index mismatch)
I20240502 14:00:16.370887 26856 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 222. Preceding OpId from leader: term: 1 index: 223. (index mismatch)
I20240502 14:00:16.371086 27117 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 222. Preceding OpId from leader: term: 1 index: 223. (index mismatch)
I20240502 14:00:16.371169 28019 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 223, Last known committed idx: 222, Time since last communication: 0.000s
I20240502 14:00:16.371310 29770 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 223, Last known committed idx: 222, Time since last communication: 0.000s
I20240502 14:00:16.371683 29770 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 223, Last known committed idx: 222, Time since last communication: 0.000s
I20240502 14:00:16.372440 28019 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 223, Last known committed idx: 222, Time since last communication: 0.000s
I20240502 14:00:16.373196 29770 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 LEADER]: Committing config change with OpId 1.223: config changed from index 75 to 223, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 223 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:16.373620 28365 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.223: config changed from index 75 to 223, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 223 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:16.375586 27254 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.223: config changed from index 75 to 223, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 223 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:16.375841 26856 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.223: config changed from index 75 to 223, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 223 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:16.376710 26709 catalog_manager.cc:5484] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 reported cstate change: config changed from index 75 to 223, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" committed_config { opid_index: 223 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:16.381210 26927 consensus_peers.cc:485] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 -> Peer b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72:46715): Couldn't send request to peer b15496b8de1b4bc084cdfbb7c581f361. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 41e73be38c4348a5bb86da6c0f6c6d22. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:16.382218 27117 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.223: config changed from index 75 to 223, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 223 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:16.387066 27381 consensus_queue.cc:237] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 262, Committed index: 262, Last appended: 1.262, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 263 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:16.388799 27519 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 262. Preceding OpId from leader: term: 1 index: 263. (index mismatch)
I20240502 14:00:16.388878 28365 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 262. Preceding OpId from leader: term: 1 index: 263. (index mismatch)
I20240502 14:00:16.388880 26856 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 262. Preceding OpId from leader: term: 1 index: 263. (index mismatch)
I20240502 14:00:16.389463 27117 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 262. Preceding OpId from leader: term: 1 index: 263. (index mismatch)
I20240502 14:00:16.389272 29785 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 263, Last known committed idx: 262, Time since last communication: 0.001s
I20240502 14:00:16.390300 29705 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 263, Last known committed idx: 262, Time since last communication: 0.000s
I20240502 14:00:16.390471 29788 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 263, Last known committed idx: 262, Time since last communication: 0.000s
I20240502 14:00:16.390705 29462 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 263, Last known committed idx: 262, Time since last communication: 0.000s
I20240502 14:00:16.392885 29705 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [term 1 LEADER]: Committing config change with OpId 1.263: config changed from index 142 to 263, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 263 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:16.393246 28365 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.263: config changed from index 142 to 263, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 263 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:16.393738 27519 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.263: config changed from index 142 to 263, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 263 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
W20240502 14:00:16.395068 27323 consensus_peers.cc:485] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 -> Peer b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72:46715): Couldn't send request to peer b15496b8de1b4bc084cdfbb7c581f361. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f7ea57a2ffe04e9e962c61d0e82fc475. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:16.395757 26710 catalog_manager.cc:5484] T f7ea57a2ffe04e9e962c61d0e82fc475 P 5f9ddfa1f6fc484d8fb96075a8efb649 reported cstate change: config changed from index 142 to 263, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "e44f8c16a14a48a38a5a74af2899c217" committed_config { opid_index: 263 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:16.395854 27117 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.263: config changed from index 142 to 263, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 263 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:16.400763 26856 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.263: config changed from index 142 to 263, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 263 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:16.404831 26856 consensus_queue.cc:237] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 292, Committed index: 292, Last appended: 1.292, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:16.406601 27519 raft_consensus.cc:1261] T 53a68b75cbce41d69432f21f2cfbf256 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer d3e36186faa442b7bd87357e74d47f68: Log matching property violated. Preceding OpId in replica: term: 1 index: 292. Preceding OpId from leader: term: 1 index: 293. (index mismatch)
I20240502 14:00:16.406723 27117 raft_consensus.cc:1261] T 53a68b75cbce41d69432f21f2cfbf256 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer d3e36186faa442b7bd87357e74d47f68: Log matching property violated. Preceding OpId in replica: term: 1 index: 292. Preceding OpId from leader: term: 1 index: 293. (index mismatch)
I20240502 14:00:16.406864 27254 raft_consensus.cc:1261] T 53a68b75cbce41d69432f21f2cfbf256 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer d3e36186faa442b7bd87357e74d47f68: Log matching property violated. Preceding OpId in replica: term: 1 index: 292. Preceding OpId from leader: term: 1 index: 293. (index mismatch)
I20240502 14:00:16.407088 28436 consensus_queue.cc:1035] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 293, Last known committed idx: 292, Time since last communication: 0.000s
I20240502 14:00:16.407187 26985 raft_consensus.cc:1261] T 53a68b75cbce41d69432f21f2cfbf256 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer d3e36186faa442b7bd87357e74d47f68: Log matching property violated. Preceding OpId in replica: term: 1 index: 292. Preceding OpId from leader: term: 1 index: 293. (index mismatch)
I20240502 14:00:16.407670 28436 consensus_queue.cc:1035] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 293, Last known committed idx: 292, Time since last communication: 0.000s
I20240502 14:00:16.407984 28436 consensus_queue.cc:1035] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 293, Last known committed idx: 292, Time since last communication: 0.000s
I20240502 14:00:16.409351 29803 consensus_queue.cc:1035] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 293, Last known committed idx: 292, Time since last communication: 0.000s
I20240502 14:00:16.409972 28436 raft_consensus.cc:2932] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [term 1 LEADER]: Committing config change with OpId 1.293: config changed from index -1 to 293, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:16.410305 26985 raft_consensus.cc:2932] T 53a68b75cbce41d69432f21f2cfbf256 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.293: config changed from index -1 to 293, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:16.412065 27117 raft_consensus.cc:2932] T 53a68b75cbce41d69432f21f2cfbf256 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.293: config changed from index -1 to 293, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:16.412245 27254 raft_consensus.cc:2932] T 53a68b75cbce41d69432f21f2cfbf256 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.293: config changed from index -1 to 293, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:16.412379 26713 catalog_manager.cc:5484] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 reported cstate change: config changed from index -1 to 293, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "d3e36186faa442b7bd87357e74d47f68" committed_config { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:16.413195 26795 consensus_peers.cc:485] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 -> Peer b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72:46715): Couldn't send request to peer b15496b8de1b4bc084cdfbb7c581f361. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 53a68b75cbce41d69432f21f2cfbf256. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:16.413148 27519 raft_consensus.cc:2932] T 53a68b75cbce41d69432f21f2cfbf256 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.293: config changed from index -1 to 293, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:16.422745 27381 consensus_queue.cc:237] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 151, Committed index: 151, Last appended: 1.151, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:16.424199 27117 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 151. Preceding OpId from leader: term: 1 index: 152. (index mismatch)
W20240502 14:00:16.424371 27323 consensus_peers.cc:485] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 -> Peer b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72:46715): Couldn't send request to peer b15496b8de1b4bc084cdfbb7c581f361. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 945d054a56ed4043b7b49d14ed917532. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:16.424458 27519 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 151. Preceding OpId from leader: term: 1 index: 152. (index mismatch)
I20240502 14:00:16.424546 26856 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 151. Preceding OpId from leader: term: 1 index: 152. (index mismatch)
I20240502 14:00:16.424705 28365 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 151. Preceding OpId from leader: term: 1 index: 152. (index mismatch)
I20240502 14:00:16.424885 29462 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 152, Last known committed idx: 151, Time since last communication: 0.000s
I20240502 14:00:16.425033 29786 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 152, Last known committed idx: 151, Time since last communication: 0.000s
I20240502 14:00:16.425215 29788 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 152, Last known committed idx: 151, Time since last communication: 0.000s
I20240502 14:00:16.425372 29785 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 152, Last known committed idx: 151, Time since last communication: 0.000s
I20240502 14:00:16.427350 29462 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [term 1 LEADER]: Committing config change with OpId 1.152: config changed from index 62 to 152, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:16.428028 27519 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.152: config changed from index 62 to 152, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:16.428387 28365 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.152: config changed from index 62 to 152, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:16.428779 27117 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.152: config changed from index 62 to 152, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:16.429945 26713 catalog_manager.cc:5484] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 reported cstate change: config changed from index 62 to 152, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "e44f8c16a14a48a38a5a74af2899c217" committed_config { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:16.431386 26856 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.152: config changed from index 62 to 152, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:16.787462 29869 ts_tablet_manager.cc:902] T 41e73be38c4348a5bb86da6c0f6c6d22 P b15496b8de1b4bc084cdfbb7c581f361: Initiating tablet copy from peer 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473)
I20240502 14:00:16.788199 29869 tablet_copy_client.cc:308] T 41e73be38c4348a5bb86da6c0f6c6d22 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.66:40473
I20240502 14:00:16.791694 27012 tablet_copy_service.cc:140] P 256ce64303ec49e1a5cd8e0ffee436b0: Received BeginTabletCopySession request for tablet 41e73be38c4348a5bb86da6c0f6c6d22 from peer b15496b8de1b4bc084cdfbb7c581f361 ({username='slave'} at 127.24.66.72:59749)
I20240502 14:00:16.791802 27012 tablet_copy_service.cc:161] P 256ce64303ec49e1a5cd8e0ffee436b0: Beginning new tablet copy session on tablet 41e73be38c4348a5bb86da6c0f6c6d22 from peer b15496b8de1b4bc084cdfbb7c581f361 at {username='slave'} at 127.24.66.72:59749: session id = b15496b8de1b4bc084cdfbb7c581f361-41e73be38c4348a5bb86da6c0f6c6d22
I20240502 14:00:16.792785 27012 tablet_copy_source_session.cc:205] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:16.795713 29869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41e73be38c4348a5bb86da6c0f6c6d22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:16.800460 29869 tablet_copy_client.cc:783] T 41e73be38c4348a5bb86da6c0f6c6d22 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:16.800637 29869 tablet_copy_client.cc:647] T 41e73be38c4348a5bb86da6c0f6c6d22 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:16.802295 29869 tablet_copy_client.cc:523] T 41e73be38c4348a5bb86da6c0f6c6d22 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:16.803871 29869 tablet_bootstrap.cc:492] T 41e73be38c4348a5bb86da6c0f6c6d22 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap starting.
I20240502 14:00:16.821066 29869 log.cc:824] T 41e73be38c4348a5bb86da6c0f6c6d22 P b15496b8de1b4bc084cdfbb7c581f361: Log is configured to *not* fsync() on all Append() calls
I20240502 14:00:16.853440 29876 ts_tablet_manager.cc:902] T 53a68b75cbce41d69432f21f2cfbf256 P b15496b8de1b4bc084cdfbb7c581f361: Initiating tablet copy from peer d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841)
I20240502 14:00:16.853967 29876 tablet_copy_client.cc:308] T 53a68b75cbce41d69432f21f2cfbf256 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.65:40841
I20240502 14:00:16.857743 26880 tablet_copy_service.cc:140] P d3e36186faa442b7bd87357e74d47f68: Received BeginTabletCopySession request for tablet 53a68b75cbce41d69432f21f2cfbf256 from peer b15496b8de1b4bc084cdfbb7c581f361 ({username='slave'} at 127.24.66.72:37129)
I20240502 14:00:16.857875 26880 tablet_copy_service.cc:161] P d3e36186faa442b7bd87357e74d47f68: Beginning new tablet copy session on tablet 53a68b75cbce41d69432f21f2cfbf256 from peer b15496b8de1b4bc084cdfbb7c581f361 at {username='slave'} at 127.24.66.72:37129: session id = b15496b8de1b4bc084cdfbb7c581f361-53a68b75cbce41d69432f21f2cfbf256
I20240502 14:00:16.858866 26880 tablet_copy_source_session.cc:205] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:16.859843 29876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53a68b75cbce41d69432f21f2cfbf256. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:16.861570 29876 tablet_copy_client.cc:783] T 53a68b75cbce41d69432f21f2cfbf256 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:16.861717 29876 tablet_copy_client.cc:647] T 53a68b75cbce41d69432f21f2cfbf256 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:16.863014 29882 ts_tablet_manager.cc:902] T f7ea57a2ffe04e9e962c61d0e82fc475 P b15496b8de1b4bc084cdfbb7c581f361: Initiating tablet copy from peer e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107)
I20240502 14:00:16.863570 29882 tablet_copy_client.cc:308] T f7ea57a2ffe04e9e962c61d0e82fc475 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.69:39107
I20240502 14:00:16.863909 29876 tablet_copy_client.cc:523] T 53a68b75cbce41d69432f21f2cfbf256 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:16.865145 29876 tablet_bootstrap.cc:492] T 53a68b75cbce41d69432f21f2cfbf256 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap starting.
I20240502 14:00:16.867604 27408 tablet_copy_service.cc:140] P e44f8c16a14a48a38a5a74af2899c217: Received BeginTabletCopySession request for tablet f7ea57a2ffe04e9e962c61d0e82fc475 from peer b15496b8de1b4bc084cdfbb7c581f361 ({username='slave'} at 127.24.66.72:32821)
I20240502 14:00:16.867717 27408 tablet_copy_service.cc:161] P e44f8c16a14a48a38a5a74af2899c217: Beginning new tablet copy session on tablet f7ea57a2ffe04e9e962c61d0e82fc475 from peer b15496b8de1b4bc084cdfbb7c581f361 at {username='slave'} at 127.24.66.72:32821: session id = b15496b8de1b4bc084cdfbb7c581f361-f7ea57a2ffe04e9e962c61d0e82fc475
I20240502 14:00:16.868713 27408 tablet_copy_source_session.cc:205] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:16.869544 29882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7ea57a2ffe04e9e962c61d0e82fc475. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:16.871901 29882 tablet_copy_client.cc:783] T f7ea57a2ffe04e9e962c61d0e82fc475 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:16.872056 29882 tablet_copy_client.cc:647] T f7ea57a2ffe04e9e962c61d0e82fc475 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:16.873842 29887 ts_tablet_manager.cc:902] T 1ad8b348f02043dfa4736276dc306895 P b15496b8de1b4bc084cdfbb7c581f361: Initiating tablet copy from peer b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68:35167)
I20240502 14:00:16.873972 29882 tablet_copy_client.cc:523] T f7ea57a2ffe04e9e962c61d0e82fc475 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:16.874447 29887 tablet_copy_client.cc:308] T 1ad8b348f02043dfa4736276dc306895 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:35167
I20240502 14:00:16.875265 29882 tablet_bootstrap.cc:492] T f7ea57a2ffe04e9e962c61d0e82fc475 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap starting.
I20240502 14:00:16.879339 27275 tablet_copy_service.cc:140] P b4fc3c795b1d4505b3a2903532d38b52: Received BeginTabletCopySession request for tablet 1ad8b348f02043dfa4736276dc306895 from peer b15496b8de1b4bc084cdfbb7c581f361 ({username='slave'} at 127.24.66.72:51597)
I20240502 14:00:16.879436 27275 tablet_copy_service.cc:161] P b4fc3c795b1d4505b3a2903532d38b52: Beginning new tablet copy session on tablet 1ad8b348f02043dfa4736276dc306895 from peer b15496b8de1b4bc084cdfbb7c581f361 at {username='slave'} at 127.24.66.72:51597: session id = b15496b8de1b4bc084cdfbb7c581f361-1ad8b348f02043dfa4736276dc306895
I20240502 14:00:16.880321 27275 tablet_copy_source_session.cc:205] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:16.881278 29887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ad8b348f02043dfa4736276dc306895. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:16.883697 29887 tablet_copy_client.cc:783] T 1ad8b348f02043dfa4736276dc306895 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:16.883909 29887 tablet_copy_client.cc:647] T 1ad8b348f02043dfa4736276dc306895 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:16.889695 29887 tablet_copy_client.cc:523] T 1ad8b348f02043dfa4736276dc306895 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:16.890877 29887 tablet_bootstrap.cc:492] T 1ad8b348f02043dfa4736276dc306895 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap starting.
I20240502 14:00:16.962407 29655 heartbeater.cc:503] Master 127.24.66.126:45215 was elected leader, sending a full tablet report...
I20240502 14:00:16.983377 29869 tablet_bootstrap.cc:492] T 41e73be38c4348a5bb86da6c0f6c6d22 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap replayed 1/1 log segments. Stats: ops{read=223 overwritten=0 applied=223 ignored=0} inserts{seen=892 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:16.983785 29869 tablet_bootstrap.cc:492] T 41e73be38c4348a5bb86da6c0f6c6d22 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap complete.
I20240502 14:00:16.984189 29869 ts_tablet_manager.cc:1367] T 41e73be38c4348a5bb86da6c0f6c6d22 P b15496b8de1b4bc084cdfbb7c581f361: Time spent bootstrapping tablet: real 0.181s user 0.160s sys 0.004s
I20240502 14:00:16.986919 29869 raft_consensus.cc:348] T 41e73be38c4348a5bb86da6c0f6c6d22 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 223 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:16.987197 29869 raft_consensus.cc:729] T 41e73be38c4348a5bb86da6c0f6c6d22 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b15496b8de1b4bc084cdfbb7c581f361, State: Initialized, Role: LEARNER
I20240502 14:00:16.987552 29869 consensus_queue.cc:260] T 41e73be38c4348a5bb86da6c0f6c6d22 P b15496b8de1b4bc084cdfbb7c581f361 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 223, Last appended: 1.223, Last appended by leader: 223, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 223 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:16.989027 29869 ts_tablet_manager.cc:1398] T 41e73be38c4348a5bb86da6c0f6c6d22 P b15496b8de1b4bc084cdfbb7c581f361: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240502 14:00:16.991843 27012 tablet_copy_service.cc:342] P 256ce64303ec49e1a5cd8e0ffee436b0: Request end of tablet copy session b15496b8de1b4bc084cdfbb7c581f361-41e73be38c4348a5bb86da6c0f6c6d22 received from {username='slave'} at 127.24.66.72:59749
I20240502 14:00:16.991931 27012 tablet_copy_service.cc:433] P 256ce64303ec49e1a5cd8e0ffee436b0: ending tablet copy session b15496b8de1b4bc084cdfbb7c581f361-41e73be38c4348a5bb86da6c0f6c6d22 on tablet 41e73be38c4348a5bb86da6c0f6c6d22 with peer b15496b8de1b4bc084cdfbb7c581f361
I20240502 14:00:17.037218 29869 ts_tablet_manager.cc:902] T 945d054a56ed4043b7b49d14ed917532 P b15496b8de1b4bc084cdfbb7c581f361: Initiating tablet copy from peer e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107)
I20240502 14:00:17.037717 29869 tablet_copy_client.cc:308] T 945d054a56ed4043b7b49d14ed917532 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.69:39107
I20240502 14:00:17.038158 27408 tablet_copy_service.cc:140] P e44f8c16a14a48a38a5a74af2899c217: Received BeginTabletCopySession request for tablet 945d054a56ed4043b7b49d14ed917532 from peer b15496b8de1b4bc084cdfbb7c581f361 ({username='slave'} at 127.24.66.72:32821)
I20240502 14:00:17.038268 27408 tablet_copy_service.cc:161] P e44f8c16a14a48a38a5a74af2899c217: Beginning new tablet copy session on tablet 945d054a56ed4043b7b49d14ed917532 from peer b15496b8de1b4bc084cdfbb7c581f361 at {username='slave'} at 127.24.66.72:32821: session id = b15496b8de1b4bc084cdfbb7c581f361-945d054a56ed4043b7b49d14ed917532
I20240502 14:00:17.039275 27408 tablet_copy_source_session.cc:205] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:17.040100 29869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 945d054a56ed4043b7b49d14ed917532. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:17.042184 29869 tablet_copy_client.cc:783] T 945d054a56ed4043b7b49d14ed917532 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:17.042343 29869 tablet_copy_client.cc:647] T 945d054a56ed4043b7b49d14ed917532 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:17.044016 29869 tablet_copy_client.cc:523] T 945d054a56ed4043b7b49d14ed917532 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:17.045094 29869 tablet_bootstrap.cc:492] T 945d054a56ed4043b7b49d14ed917532 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap starting.
I20240502 14:00:17.101858 29876 tablet_bootstrap.cc:492] T 53a68b75cbce41d69432f21f2cfbf256 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap replayed 1/1 log segments. Stats: ops{read=293 overwritten=0 applied=293 ignored=0} inserts{seen=924 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:17.102209 29876 tablet_bootstrap.cc:492] T 53a68b75cbce41d69432f21f2cfbf256 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap complete.
I20240502 14:00:17.102430 29876 ts_tablet_manager.cc:1367] T 53a68b75cbce41d69432f21f2cfbf256 P b15496b8de1b4bc084cdfbb7c581f361: Time spent bootstrapping tablet: real 0.237s user 0.203s sys 0.016s
I20240502 14:00:17.103029 29876 raft_consensus.cc:348] T 53a68b75cbce41d69432f21f2cfbf256 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:17.103216 29876 raft_consensus.cc:729] T 53a68b75cbce41d69432f21f2cfbf256 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b15496b8de1b4bc084cdfbb7c581f361, State: Initialized, Role: LEARNER
I20240502 14:00:17.103332 29876 consensus_queue.cc:260] T 53a68b75cbce41d69432f21f2cfbf256 P b15496b8de1b4bc084cdfbb7c581f361 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 293, Last appended: 1.293, Last appended by leader: 293, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 293 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:17.103688 29876 ts_tablet_manager.cc:1398] T 53a68b75cbce41d69432f21f2cfbf256 P b15496b8de1b4bc084cdfbb7c581f361: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 14:00:17.104367 26880 tablet_copy_service.cc:342] P d3e36186faa442b7bd87357e74d47f68: Request end of tablet copy session b15496b8de1b4bc084cdfbb7c581f361-53a68b75cbce41d69432f21f2cfbf256 received from {username='slave'} at 127.24.66.72:37129
I20240502 14:00:17.104483 26880 tablet_copy_service.cc:433] P d3e36186faa442b7bd87357e74d47f68: ending tablet copy session b15496b8de1b4bc084cdfbb7c581f361-53a68b75cbce41d69432f21f2cfbf256 on tablet 53a68b75cbce41d69432f21f2cfbf256 with peer b15496b8de1b4bc084cdfbb7c581f361
I20240502 14:00:17.147454 29882 tablet_bootstrap.cc:492] T f7ea57a2ffe04e9e962c61d0e82fc475 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap replayed 1/1 log segments. Stats: ops{read=263 overwritten=0 applied=263 ignored=0} inserts{seen=891 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:17.147816 29882 tablet_bootstrap.cc:492] T f7ea57a2ffe04e9e962c61d0e82fc475 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap complete.
I20240502 14:00:17.148049 29882 ts_tablet_manager.cc:1367] T f7ea57a2ffe04e9e962c61d0e82fc475 P b15496b8de1b4bc084cdfbb7c581f361: Time spent bootstrapping tablet: real 0.273s user 0.185s sys 0.016s
I20240502 14:00:17.148550 29882 raft_consensus.cc:348] T f7ea57a2ffe04e9e962c61d0e82fc475 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 263 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:17.148694 29882 raft_consensus.cc:729] T f7ea57a2ffe04e9e962c61d0e82fc475 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b15496b8de1b4bc084cdfbb7c581f361, State: Initialized, Role: LEARNER
I20240502 14:00:17.148803 29882 consensus_queue.cc:260] T f7ea57a2ffe04e9e962c61d0e82fc475 P b15496b8de1b4bc084cdfbb7c581f361 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 263, Last appended: 1.263, Last appended by leader: 263, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 263 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:17.149261 29882 ts_tablet_manager.cc:1398] T f7ea57a2ffe04e9e962c61d0e82fc475 P b15496b8de1b4bc084cdfbb7c581f361: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 14:00:17.149871 27408 tablet_copy_service.cc:342] P e44f8c16a14a48a38a5a74af2899c217: Request end of tablet copy session b15496b8de1b4bc084cdfbb7c581f361-f7ea57a2ffe04e9e962c61d0e82fc475 received from {username='slave'} at 127.24.66.72:32821
I20240502 14:00:17.149962 27408 tablet_copy_service.cc:433] P e44f8c16a14a48a38a5a74af2899c217: ending tablet copy session b15496b8de1b4bc084cdfbb7c581f361-f7ea57a2ffe04e9e962c61d0e82fc475 on tablet f7ea57a2ffe04e9e962c61d0e82fc475 with peer b15496b8de1b4bc084cdfbb7c581f361
I20240502 14:00:17.164201 29869 tablet_bootstrap.cc:492] T 945d054a56ed4043b7b49d14ed917532 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap replayed 1/1 log segments. Stats: ops{read=152 overwritten=0 applied=152 ignored=0} inserts{seen=780 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:17.164496 29869 tablet_bootstrap.cc:492] T 945d054a56ed4043b7b49d14ed917532 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap complete.
I20240502 14:00:17.164680 29869 ts_tablet_manager.cc:1367] T 945d054a56ed4043b7b49d14ed917532 P b15496b8de1b4bc084cdfbb7c581f361: Time spent bootstrapping tablet: real 0.120s user 0.106s sys 0.008s
I20240502 14:00:17.165055 29869 raft_consensus.cc:348] T 945d054a56ed4043b7b49d14ed917532 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:17.165155 29869 raft_consensus.cc:729] T 945d054a56ed4043b7b49d14ed917532 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b15496b8de1b4bc084cdfbb7c581f361, State: Initialized, Role: LEARNER
I20240502 14:00:17.165259 29869 consensus_queue.cc:260] T 945d054a56ed4043b7b49d14ed917532 P b15496b8de1b4bc084cdfbb7c581f361 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 152, Last appended: 1.152, Last appended by leader: 152, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 152 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:17.165520 29869 ts_tablet_manager.cc:1398] T 945d054a56ed4043b7b49d14ed917532 P b15496b8de1b4bc084cdfbb7c581f361: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:17.165968 27408 tablet_copy_service.cc:342] P e44f8c16a14a48a38a5a74af2899c217: Request end of tablet copy session b15496b8de1b4bc084cdfbb7c581f361-945d054a56ed4043b7b49d14ed917532 received from {username='slave'} at 127.24.66.72:32821
I20240502 14:00:17.166069 27408 tablet_copy_service.cc:433] P e44f8c16a14a48a38a5a74af2899c217: ending tablet copy session b15496b8de1b4bc084cdfbb7c581f361-945d054a56ed4043b7b49d14ed917532 on tablet 945d054a56ed4043b7b49d14ed917532 with peer b15496b8de1b4bc084cdfbb7c581f361
I20240502 14:00:17.201792 29887 tablet_bootstrap.cc:492] T 1ad8b348f02043dfa4736276dc306895 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap replayed 1/1 log segments. Stats: ops{read=351 overwritten=0 applied=351 ignored=0} inserts{seen=1023 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:17.202102 29887 tablet_bootstrap.cc:492] T 1ad8b348f02043dfa4736276dc306895 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap complete.
I20240502 14:00:17.202334 29887 ts_tablet_manager.cc:1367] T 1ad8b348f02043dfa4736276dc306895 P b15496b8de1b4bc084cdfbb7c581f361: Time spent bootstrapping tablet: real 0.311s user 0.222s sys 0.024s
I20240502 14:00:17.202755 29887 raft_consensus.cc:348] T 1ad8b348f02043dfa4736276dc306895 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 351 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:17.202872 29887 raft_consensus.cc:729] T 1ad8b348f02043dfa4736276dc306895 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b15496b8de1b4bc084cdfbb7c581f361, State: Initialized, Role: LEARNER
I20240502 14:00:17.202980 29887 consensus_queue.cc:260] T 1ad8b348f02043dfa4736276dc306895 P b15496b8de1b4bc084cdfbb7c581f361 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 351, Last appended: 1.351, Last appended by leader: 351, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 351 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:17.203238 29887 ts_tablet_manager.cc:1398] T 1ad8b348f02043dfa4736276dc306895 P b15496b8de1b4bc084cdfbb7c581f361: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:17.203747 27275 tablet_copy_service.cc:342] P b4fc3c795b1d4505b3a2903532d38b52: Request end of tablet copy session b15496b8de1b4bc084cdfbb7c581f361-1ad8b348f02043dfa4736276dc306895 received from {username='slave'} at 127.24.66.72:51597
I20240502 14:00:17.203848 27275 tablet_copy_service.cc:433] P b4fc3c795b1d4505b3a2903532d38b52: ending tablet copy session b15496b8de1b4bc084cdfbb7c581f361-1ad8b348f02043dfa4736276dc306895 on tablet 1ad8b348f02043dfa4736276dc306895 with peer b15496b8de1b4bc084cdfbb7c581f361
I20240502 14:00:17.304121 29609 raft_consensus.cc:1203] T 1ad8b348f02043dfa4736276dc306895 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.350->[1.351-1.351] Dedup: 1.351->[]
I20240502 14:00:17.321254 29609 raft_consensus.cc:1203] T f7ea57a2ffe04e9e962c61d0e82fc475 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.262->[1.263-1.263] Dedup: 1.263->[]
I20240502 14:00:17.341722 29609 raft_consensus.cc:1203] T 41e73be38c4348a5bb86da6c0f6c6d22 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.222->[1.223-1.223] Dedup: 1.223->[]
I20240502 14:00:17.451109 29609 raft_consensus.cc:1203] T 53a68b75cbce41d69432f21f2cfbf256 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.292->[1.293-1.293] Dedup: 1.293->[]
I20240502 14:00:17.657963 29609 raft_consensus.cc:1203] T 945d054a56ed4043b7b49d14ed917532 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.151->[1.152-1.152] Dedup: 1.152->[]
I20240502 14:00:17.680137 29952 raft_consensus.cc:1050] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52: attempting to promote NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 to VOTER
I20240502 14:00:17.680626 29952 consensus_queue.cc:237] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 351, Committed index: 351, Last appended: 1.351, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 352 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } }
I20240502 14:00:17.683228 27117 raft_consensus.cc:1261] T 1ad8b348f02043dfa4736276dc306895 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 351. Preceding OpId from leader: term: 1 index: 352. (index mismatch)
I20240502 14:00:17.684069 26856 raft_consensus.cc:1261] T 1ad8b348f02043dfa4736276dc306895 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 351. Preceding OpId from leader: term: 1 index: 352. (index mismatch)
I20240502 14:00:17.684612 29609 raft_consensus.cc:1261] T 1ad8b348f02043dfa4736276dc306895 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 351. Preceding OpId from leader: term: 1 index: 352. (index mismatch)
I20240502 14:00:17.685034 29995 consensus_queue.cc:1035] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 352, Last known committed idx: 351, Time since last communication: 0.000s
I20240502 14:00:17.685245 27381 raft_consensus.cc:1261] T 1ad8b348f02043dfa4736276dc306895 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 351. Preceding OpId from leader: term: 1 index: 352. (index mismatch)
I20240502 14:00:17.685369 29995 consensus_queue.cc:1035] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 352, Last known committed idx: 351, Time since last communication: 0.000s
I20240502 14:00:17.685554 26985 raft_consensus.cc:1261] T 1ad8b348f02043dfa4736276dc306895 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 351. Preceding OpId from leader: term: 1 index: 352. (index mismatch)
I20240502 14:00:17.685616 29995 consensus_queue.cc:1035] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 352, Last known committed idx: 351, Time since last communication: 0.000s
I20240502 14:00:17.685724 29951 consensus_queue.cc:1035] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 352, Last known committed idx: 351, Time since last communication: 0.000s
I20240502 14:00:17.685993 29952 consensus_queue.cc:1035] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 352, Last known committed idx: 351, Time since last communication: 0.000s
I20240502 14:00:17.688743 29993 raft_consensus.cc:2932] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 LEADER]: Committing config change with OpId 1.352: config changed from index 351 to 352, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 352 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.689175 27381 raft_consensus.cc:2932] T 1ad8b348f02043dfa4736276dc306895 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.352: config changed from index 351 to 352, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 352 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.691299 26858 raft_consensus.cc:2932] T 1ad8b348f02043dfa4736276dc306895 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.352: config changed from index 351 to 352, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 352 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.692315 26708 catalog_manager.cc:5484] T 1ad8b348f02043dfa4736276dc306895 P e44f8c16a14a48a38a5a74af2899c217 reported cstate change: config changed from index 351 to 352, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b4fc3c795b1d4505b3a2903532d38b52" committed_config { opid_index: 352 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.693454 26985 raft_consensus.cc:2932] T 1ad8b348f02043dfa4736276dc306895 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.352: config changed from index 351 to 352, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 352 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.693609 29609 raft_consensus.cc:2932] T 1ad8b348f02043dfa4736276dc306895 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Committing config change with OpId 1.352: config changed from index 351 to 352, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 352 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.695346 27117 raft_consensus.cc:2932] T 1ad8b348f02043dfa4736276dc306895 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.352: config changed from index 351 to 352, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 352 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.702418 27254 consensus_queue.cc:237] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 352, Committed index: 352, Last appended: 1.352, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 353 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } }
I20240502 14:00:17.703855 26985 raft_consensus.cc:1261] T 1ad8b348f02043dfa4736276dc306895 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 352. Preceding OpId from leader: term: 1 index: 353. (index mismatch)
I20240502 14:00:17.703987 26858 raft_consensus.cc:1261] T 1ad8b348f02043dfa4736276dc306895 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 352. Preceding OpId from leader: term: 1 index: 353. (index mismatch)
I20240502 14:00:17.704051 29609 raft_consensus.cc:1261] T 1ad8b348f02043dfa4736276dc306895 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 352. Preceding OpId from leader: term: 1 index: 353. (index mismatch)
I20240502 14:00:17.704457 29994 consensus_queue.cc:1035] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 353, Last known committed idx: 352, Time since last communication: 0.000s
I20240502 14:00:17.704600 29995 consensus_queue.cc:1035] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 353, Last known committed idx: 352, Time since last communication: 0.000s
I20240502 14:00:17.704696 27381 raft_consensus.cc:1261] T 1ad8b348f02043dfa4736276dc306895 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 352. Preceding OpId from leader: term: 1 index: 353. (index mismatch)
I20240502 14:00:17.704766 29952 consensus_queue.cc:1035] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 353, Last known committed idx: 352, Time since last communication: 0.000s
I20240502 14:00:17.705658 29995 consensus_queue.cc:1035] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 353, Last known committed idx: 352, Time since last communication: 0.000s
I20240502 14:00:17.706876 29952 raft_consensus.cc:2932] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 LEADER]: Committing config change with OpId 1.353: config changed from index 352 to 353, VOTER 0280081e4ff74896832ec3985c372b3a (127.24.66.67) evicted. New config: { opid_index: 353 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.707999 27381 raft_consensus.cc:2932] T 1ad8b348f02043dfa4736276dc306895 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.353: config changed from index 352 to 353, VOTER 0280081e4ff74896832ec3985c372b3a (127.24.66.67) evicted. New config: { opid_index: 353 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.709154 29609 raft_consensus.cc:2932] T 1ad8b348f02043dfa4736276dc306895 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Committing config change with OpId 1.353: config changed from index 352 to 353, VOTER 0280081e4ff74896832ec3985c372b3a (127.24.66.67) evicted. New config: { opid_index: 353 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.709533 26858 raft_consensus.cc:2932] T 1ad8b348f02043dfa4736276dc306895 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.353: config changed from index 352 to 353, VOTER 0280081e4ff74896832ec3985c372b3a (127.24.66.67) evicted. New config: { opid_index: 353 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.710135 26703 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 1ad8b348f02043dfa4736276dc306895 with cas_config_opid_index 352: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:17.711067 26712 catalog_manager.cc:5484] T 1ad8b348f02043dfa4736276dc306895 P b4fc3c795b1d4505b3a2903532d38b52 reported cstate change: config changed from index 352 to 353, VOTER 0280081e4ff74896832ec3985c372b3a (127.24.66.67) evicted. New cstate: current_term: 1 leader_uuid: "b4fc3c795b1d4505b3a2903532d38b52" committed_config { opid_index: 353 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:17.711623 26985 raft_consensus.cc:2932] T 1ad8b348f02043dfa4736276dc306895 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.353: config changed from index 352 to 353, VOTER 0280081e4ff74896832ec3985c372b3a (127.24.66.67) evicted. New config: { opid_index: 353 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.714141 27101 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ad8b348f02043dfa4736276dc306895 with delete_type TABLET_DATA_TOMBSTONED (TS 0280081e4ff74896832ec3985c372b3a not found in new config with opid_index 353) from {username='slave'} at 127.0.0.1:51610
I20240502 14:00:17.714726 30015 tablet_replica.cc:330] T 1ad8b348f02043dfa4736276dc306895 P 0280081e4ff74896832ec3985c372b3a: stopping tablet replica
I20240502 14:00:17.714910 30015 raft_consensus.cc:2229] T 1ad8b348f02043dfa4736276dc306895 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:17.715078 30015 raft_consensus.cc:2258] T 1ad8b348f02043dfa4736276dc306895 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:17.716737 30015 ts_tablet_manager.cc:1875] T 1ad8b348f02043dfa4736276dc306895 P 0280081e4ff74896832ec3985c372b3a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:17.718988 30015 ts_tablet_manager.cc:1888] T 1ad8b348f02043dfa4736276dc306895 P 0280081e4ff74896832ec3985c372b3a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.352
I20240502 14:00:17.719077 30015 log.cc:1196] T 1ad8b348f02043dfa4736276dc306895 P 0280081e4ff74896832ec3985c372b3a: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/wals/1ad8b348f02043dfa4736276dc306895
I20240502 14:00:17.719696 26703 catalog_manager.cc:4830] TS 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733): tablet 1ad8b348f02043dfa4736276dc306895 (table rebalance_test_table_3 [id=ecda844ae2ee4cb5a179b79c8dacb5ac]) successfully deleted
I20240502 14:00:17.730425 27515 consensus_queue.cc:237] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 288, Committed index: 288, Last appended: 1.288, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 289 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:17.732584 27381 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 288. Preceding OpId from leader: term: 1 index: 289. (index mismatch)
I20240502 14:00:17.732947 27254 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 288. Preceding OpId from leader: term: 1 index: 289. (index mismatch)
I20240502 14:00:17.733004 27117 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 288. Preceding OpId from leader: term: 1 index: 289. (index mismatch)
I20240502 14:00:17.732990 30028 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 289, Last known committed idx: 288, Time since last communication: 0.000s
I20240502 14:00:17.733574 29459 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 289, Last known committed idx: 288, Time since last communication: 0.000s
I20240502 14:00:17.733886 29459 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 289, Last known committed idx: 288, Time since last communication: 0.000s
I20240502 14:00:17.733996 28365 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 288. Preceding OpId from leader: term: 1 index: 289. (index mismatch)
I20240502 14:00:17.734416 30029 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 289, Last known committed idx: 288, Time since last communication: 0.000s
W20240502 14:00:17.736896 27455 consensus_peers.cc:485] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 -> Peer b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72:46715): Couldn't send request to peer b15496b8de1b4bc084cdfbb7c581f361. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9efaa3c68c534606bd37bbe4703b2ab2. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:17.737517 30030 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [term 1 LEADER]: Committing config change with OpId 1.289: config changed from index 87 to 289, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 289 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:17.737768 29955 raft_consensus.cc:1050] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0: attempting to promote NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 to VOTER
I20240502 14:00:17.738176 29955 consensus_queue.cc:237] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 223, Committed index: 223, Last appended: 1.223, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 224 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } }
I20240502 14:00:17.738330 27381 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.289: config changed from index 87 to 289, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 289 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:17.740550 28365 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 223. Preceding OpId from leader: term: 1 index: 224. (index mismatch)
I20240502 14:00:17.740298 27117 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.289: config changed from index 87 to 289, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 289 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:17.740973 29770 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 224, Last known committed idx: 223, Time since last communication: 0.000s
I20240502 14:00:17.740972 26712 catalog_manager.cc:5484] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 reported cstate change: config changed from index 87 to 289, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "34b161bd719d4d72baf8da4f40aef136" committed_config { opid_index: 289 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:17.742178 26858 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 223. Preceding OpId from leader: term: 1 index: 224. (index mismatch)
I20240502 14:00:17.742641 27254 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 223. Preceding OpId from leader: term: 1 index: 224. (index mismatch)
I20240502 14:00:17.743556 29770 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 224, Last known committed idx: 223, Time since last communication: 0.000s
I20240502 14:00:17.745378 27117 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 223. Preceding OpId from leader: term: 1 index: 224. (index mismatch)
I20240502 14:00:17.745445 27252 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.289: config changed from index 87 to 289, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 289 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:17.745751 29609 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 223. Preceding OpId from leader: term: 1 index: 224. (index mismatch)
I20240502 14:00:17.746011 29770 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 224, Last known committed idx: 223, Time since last communication: 0.000s
I20240502 14:00:17.746290 28365 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.289: config changed from index 87 to 289, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 289 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:17.748169 29770 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 224, Last known committed idx: 223, Time since last communication: 0.000s
I20240502 14:00:17.748749 29770 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 224, Last known committed idx: 223, Time since last communication: 0.000s
I20240502 14:00:17.751053 29770 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 LEADER]: Committing config change with OpId 1.224: config changed from index 223 to 224, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 224 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.751570 29609 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Committing config change with OpId 1.224: config changed from index 223 to 224, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 224 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.753424 28365 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.224: config changed from index 223 to 224, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 224 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.753489 26858 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.224: config changed from index 223 to 224, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 224 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.753757 26713 catalog_manager.cc:5484] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 reported cstate change: config changed from index 223 to 224, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" committed_config { opid_index: 224 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:17.759438 27117 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.224: config changed from index 223 to 224, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 224 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.763577 26985 consensus_queue.cc:237] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 224, Committed index: 224, Last appended: 1.224, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 225 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } }
I20240502 14:00:17.764430 27254 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.224: config changed from index 223 to 224, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 224 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.765661 26858 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 224. Preceding OpId from leader: term: 1 index: 225. (index mismatch)
I20240502 14:00:17.765836 29609 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 224. Preceding OpId from leader: term: 1 index: 225. (index mismatch)
I20240502 14:00:17.766218 28365 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 224. Preceding OpId from leader: term: 1 index: 225. (index mismatch)
I20240502 14:00:17.766650 30043 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 225, Last known committed idx: 224, Time since last communication: 0.000s
I20240502 14:00:17.766981 30043 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 225, Last known committed idx: 224, Time since last communication: 0.000s
I20240502 14:00:17.767578 30044 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 225, Last known committed idx: 224, Time since last communication: 0.000s
I20240502 14:00:17.768198 27117 raft_consensus.cc:1261] T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer 256ce64303ec49e1a5cd8e0ffee436b0: Log matching property violated. Preceding OpId in replica: term: 1 index: 224. Preceding OpId from leader: term: 1 index: 225. (index mismatch)
I20240502 14:00:17.768937 30044 consensus_queue.cc:1035] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 225, Last known committed idx: 224, Time since last communication: 0.000s
I20240502 14:00:17.770767 30043 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 LEADER]: Committing config change with OpId 1.225: config changed from index 224 to 225, VOTER b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68) evicted. New config: { opid_index: 225 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.771723 26858 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.225: config changed from index 224 to 225, VOTER b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68) evicted. New config: { opid_index: 225 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.772853 28365 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.225: config changed from index 224 to 225, VOTER b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68) evicted. New config: { opid_index: 225 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.773612 29609 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Committing config change with OpId 1.225: config changed from index 224 to 225, VOTER b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68) evicted. New config: { opid_index: 225 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.774174 27117 raft_consensus.cc:2932] T 41e73be38c4348a5bb86da6c0f6c6d22 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.225: config changed from index 224 to 225, VOTER b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68) evicted. New config: { opid_index: 225 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.774382 26700 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 41e73be38c4348a5bb86da6c0f6c6d22 with cas_config_opid_index 224: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:17.774426 26714 catalog_manager.cc:5484] T 41e73be38c4348a5bb86da6c0f6c6d22 P d3e36186faa442b7bd87357e74d47f68 reported cstate change: config changed from index 224 to 225, VOTER b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" committed_config { opid_index: 225 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.777781 27234 tablet_service.cc:1514] Processing DeleteTablet for tablet 41e73be38c4348a5bb86da6c0f6c6d22 with delete_type TABLET_DATA_TOMBSTONED (TS b4fc3c795b1d4505b3a2903532d38b52 not found in new config with opid_index 225) from {username='slave'} at 127.0.0.1:57834
I20240502 14:00:17.779315 30068 tablet_replica.cc:330] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52: stopping tablet replica
I20240502 14:00:17.779505 30068 raft_consensus.cc:2229] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:17.779727 30068 raft_consensus.cc:2258] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:17.781453 30068 ts_tablet_manager.cc:1875] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:17.783708 30068 ts_tablet_manager.cc:1888] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.224
I20240502 14:00:17.783792 30068 log.cc:1196] T 41e73be38c4348a5bb86da6c0f6c6d22 P b4fc3c795b1d4505b3a2903532d38b52: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/41e73be38c4348a5bb86da6c0f6c6d22
I20240502 14:00:17.784380 26703 catalog_manager.cc:4830] TS b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68:35167): tablet 41e73be38c4348a5bb86da6c0f6c6d22 (table rebalance_test_table_1 [id=a35cd53b479c486fad6a93e2f4317542]) successfully deleted
I20240502 14:00:17.798645 27254 consensus_queue.cc:237] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 318, Committed index: 318, Last appended: 1.318, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 319 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false replace: true } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:17.800370 28365 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 318. Preceding OpId from leader: term: 1 index: 319. (index mismatch)
I20240502 14:00:17.800454 27117 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 318. Preceding OpId from leader: term: 1 index: 319. (index mismatch)
I20240502 14:00:17.800570 26985 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 318. Preceding OpId from leader: term: 1 index: 319. (index mismatch)
I20240502 14:00:17.800745 27381 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 318. Preceding OpId from leader: term: 1 index: 319. (index mismatch)
I20240502 14:00:17.800943 29993 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 319, Last known committed idx: 318, Time since last communication: 0.000s
W20240502 14:00:17.801126 27191 consensus_peers.cc:485] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 -> Peer b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72:46715): Couldn't send request to peer b15496b8de1b4bc084cdfbb7c581f361. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f4bb31e5c0454e54a7b3855445120783. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:17.801316 29951 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 319, Last known committed idx: 318, Time since last communication: 0.000s
I20240502 14:00:17.801458 29750 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 319, Last known committed idx: 318, Time since last communication: 0.000s
I20240502 14:00:17.801580 29952 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 319, Last known committed idx: 318, Time since last communication: 0.000s
I20240502 14:00:17.803594 29951 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 LEADER]: Committing config change with OpId 1.319: config changed from index 168 to 319, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 319 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false replace: true } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:17.804311 27117 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.319: config changed from index 168 to 319, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 319 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false replace: true } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:17.804656 27381 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.319: config changed from index 168 to 319, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 319 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false replace: true } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:17.806531 26713 catalog_manager.cc:5484] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 reported cstate change: config changed from index 168 to 319, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "b4fc3c795b1d4505b3a2903532d38b52" committed_config { opid_index: 319 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:17.806723 28365 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.319: config changed from index 168 to 319, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 319 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false replace: true } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:17.809190 26985 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.319: config changed from index 168 to 319, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 319 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false replace: true } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:17.855536 29953 raft_consensus.cc:1050] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217: attempting to promote NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 to VOTER
I20240502 14:00:17.856010 29953 consensus_queue.cc:237] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 263, Committed index: 263, Last appended: 1.263, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 264 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } }
I20240502 14:00:17.857741 27515 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 263. Preceding OpId from leader: term: 1 index: 264. (index mismatch)
I20240502 14:00:17.857762 28365 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 263. Preceding OpId from leader: term: 1 index: 264. (index mismatch)
I20240502 14:00:17.857935 26858 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 263. Preceding OpId from leader: term: 1 index: 264. (index mismatch)
I20240502 14:00:17.858191 29786 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 264, Last known committed idx: 263, Time since last communication: 0.000s
I20240502 14:00:17.858378 29953 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 264, Last known committed idx: 263, Time since last communication: 0.000s
I20240502 14:00:17.858557 30116 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 264, Last known committed idx: 263, Time since last communication: 0.000s
I20240502 14:00:17.859011 27117 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 263. Preceding OpId from leader: term: 1 index: 264. (index mismatch)
I20240502 14:00:17.859100 29609 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 263. Preceding OpId from leader: term: 1 index: 264. (index mismatch)
I20240502 14:00:17.860049 29786 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 264, Last known committed idx: 263, Time since last communication: 0.000s
I20240502 14:00:17.860450 29786 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 264, Last known committed idx: 263, Time since last communication: 0.000s
I20240502 14:00:17.862545 30116 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [term 1 LEADER]: Committing config change with OpId 1.264: config changed from index 263 to 264, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 264 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.862942 27117 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.264: config changed from index 263 to 264, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 264 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.862907 29609 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Committing config change with OpId 1.264: config changed from index 263 to 264, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 264 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.864838 28365 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.264: config changed from index 263 to 264, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 264 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.865597 26712 catalog_manager.cc:5484] T f7ea57a2ffe04e9e962c61d0e82fc475 P b15496b8de1b4bc084cdfbb7c581f361 reported cstate change: config changed from index 263 to 264, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e44f8c16a14a48a38a5a74af2899c217" committed_config { opid_index: 264 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.867199 27515 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.264: config changed from index 263 to 264, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 264 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.867199 26858 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.264: config changed from index 263 to 264, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 264 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.870272 27381 consensus_queue.cc:237] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 264, Committed index: 264, Last appended: 1.264, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 265 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } }
I20240502 14:00:17.871840 29609 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 264. Preceding OpId from leader: term: 1 index: 265. (index mismatch)
I20240502 14:00:17.871996 27117 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 264. Preceding OpId from leader: term: 1 index: 265. (index mismatch)
I20240502 14:00:17.872161 30116 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 265, Last known committed idx: 264, Time since last communication: 0.000s
I20240502 14:00:17.872319 28365 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 264. Preceding OpId from leader: term: 1 index: 265. (index mismatch)
I20240502 14:00:17.872435 29953 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 265, Last known committed idx: 264, Time since last communication: 0.000s
I20240502 14:00:17.872740 26858 raft_consensus.cc:1261] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 264. Preceding OpId from leader: term: 1 index: 265. (index mismatch)
I20240502 14:00:17.872859 29953 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 265, Last known committed idx: 264, Time since last communication: 0.000s
I20240502 14:00:17.873318 29953 consensus_queue.cc:1035] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 265, Last known committed idx: 264, Time since last communication: 0.000s
I20240502 14:00:17.874363 30116 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 [term 1 LEADER]: Committing config change with OpId 1.265: config changed from index 264 to 265, VOTER 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70) evicted. New config: { opid_index: 265 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.875278 28365 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.265: config changed from index 264 to 265, VOTER 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70) evicted. New config: { opid_index: 265 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.876142 26701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f7ea57a2ffe04e9e962c61d0e82fc475 with cas_config_opid_index 264: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:17.876116 27117 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.265: config changed from index 264 to 265, VOTER 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70) evicted. New config: { opid_index: 265 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.876505 26858 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.265: config changed from index 264 to 265, VOTER 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70) evicted. New config: { opid_index: 265 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.876559 29609 raft_consensus.cc:2932] T f7ea57a2ffe04e9e962c61d0e82fc475 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Committing config change with OpId 1.265: config changed from index 264 to 265, VOTER 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70) evicted. New config: { opid_index: 265 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.877014 26714 catalog_manager.cc:5484] T f7ea57a2ffe04e9e962c61d0e82fc475 P e44f8c16a14a48a38a5a74af2899c217 reported cstate change: config changed from index 264 to 265, VOTER 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70) evicted. New cstate: current_term: 1 leader_uuid: "e44f8c16a14a48a38a5a74af2899c217" committed_config { opid_index: 265 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:17.879797 27499 tablet_service.cc:1514] Processing DeleteTablet for tablet f7ea57a2ffe04e9e962c61d0e82fc475 with delete_type TABLET_DATA_TOMBSTONED (TS 34b161bd719d4d72baf8da4f40aef136 not found in new config with opid_index 265) from {username='slave'} at 127.0.0.1:33294
I20240502 14:00:17.881305 30122 tablet_replica.cc:330] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136: stopping tablet replica
I20240502 14:00:17.881464 30122 raft_consensus.cc:2229] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:17.881772 30122 raft_consensus.cc:2258] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:17.883049 30122 ts_tablet_manager.cc:1875] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:17.884768 30122 ts_tablet_manager.cc:1888] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.264
I20240502 14:00:17.884833 30122 log.cc:1196] T f7ea57a2ffe04e9e962c61d0e82fc475 P 34b161bd719d4d72baf8da4f40aef136: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal/wals/f7ea57a2ffe04e9e962c61d0e82fc475
I20240502 14:00:17.885412 26700 catalog_manager.cc:4830] TS 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217): tablet f7ea57a2ffe04e9e962c61d0e82fc475 (table rebalance_test_table_4 [id=56abfd3d24fd434b996a35f2174a0641]) successfully deleted
I20240502 14:00:17.944828 29961 raft_consensus.cc:1050] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68: attempting to promote NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 to VOTER
I20240502 14:00:17.945343 29961 consensus_queue.cc:237] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 293, Committed index: 293, Last appended: 1.293, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 294 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } }
I20240502 14:00:17.947296 27254 raft_consensus.cc:1261] T 53a68b75cbce41d69432f21f2cfbf256 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer d3e36186faa442b7bd87357e74d47f68: Log matching property violated. Preceding OpId in replica: term: 1 index: 293. Preceding OpId from leader: term: 1 index: 294. (index mismatch)
I20240502 14:00:17.947352 26985 raft_consensus.cc:1261] T 53a68b75cbce41d69432f21f2cfbf256 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer d3e36186faa442b7bd87357e74d47f68: Log matching property violated. Preceding OpId in replica: term: 1 index: 293. Preceding OpId from leader: term: 1 index: 294. (index mismatch)
I20240502 14:00:17.947724 27515 raft_consensus.cc:1261] T 53a68b75cbce41d69432f21f2cfbf256 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer d3e36186faa442b7bd87357e74d47f68: Log matching property violated. Preceding OpId in replica: term: 1 index: 293. Preceding OpId from leader: term: 1 index: 294. (index mismatch)
I20240502 14:00:17.947713 30124 consensus_queue.cc:1035] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 294, Last known committed idx: 293, Time since last communication: 0.000s
I20240502 14:00:17.947924 30126 consensus_queue.cc:1035] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 294, Last known committed idx: 293, Time since last communication: 0.000s
I20240502 14:00:17.948176 29609 raft_consensus.cc:1261] T 53a68b75cbce41d69432f21f2cfbf256 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Refusing update from remote peer d3e36186faa442b7bd87357e74d47f68: Log matching property violated. Preceding OpId in replica: term: 1 index: 293. Preceding OpId from leader: term: 1 index: 294. (index mismatch)
I20240502 14:00:17.948582 30126 consensus_queue.cc:1035] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 294, Last known committed idx: 293, Time since last communication: 0.000s
I20240502 14:00:17.949110 29803 consensus_queue.cc:1035] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 294, Last known committed idx: 293, Time since last communication: 0.000s
I20240502 14:00:17.951190 30126 raft_consensus.cc:2932] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [term 1 LEADER]: Committing config change with OpId 1.294: config changed from index 293 to 294, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 294 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.951666 27519 raft_consensus.cc:2932] T 53a68b75cbce41d69432f21f2cfbf256 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.294: config changed from index 293 to 294, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 294 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.953285 26985 raft_consensus.cc:2932] T 53a68b75cbce41d69432f21f2cfbf256 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.294: config changed from index 293 to 294, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 294 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.953397 27254 raft_consensus.cc:2932] T 53a68b75cbce41d69432f21f2cfbf256 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.294: config changed from index 293 to 294, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 294 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.954324 29608 raft_consensus.cc:2932] T 53a68b75cbce41d69432f21f2cfbf256 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Committing config change with OpId 1.294: config changed from index 293 to 294, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 294 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.954505 27117 raft_consensus.cc:1261] T 53a68b75cbce41d69432f21f2cfbf256 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer d3e36186faa442b7bd87357e74d47f68: Log matching property violated. Preceding OpId in replica: term: 1 index: 293. Preceding OpId from leader: term: 1 index: 294. (index mismatch)
I20240502 14:00:17.955675 30125 consensus_queue.cc:1035] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 294, Last known committed idx: 293, Time since last communication: 0.000s
I20240502 14:00:17.956527 26709 catalog_manager.cc:5484] T 53a68b75cbce41d69432f21f2cfbf256 P 34b161bd719d4d72baf8da4f40aef136 reported cstate change: config changed from index 293 to 294, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d3e36186faa442b7bd87357e74d47f68" committed_config { opid_index: 294 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:17.957669 27117 raft_consensus.cc:2932] T 53a68b75cbce41d69432f21f2cfbf256 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.294: config changed from index 293 to 294, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 294 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:18.071307 27117 consensus_queue.cc:237] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 208, Committed index: 208, Last appended: 1.208, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 209 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:18.073192 26985 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 208. Preceding OpId from leader: term: 1 index: 209. (index mismatch)
I20240502 14:00:18.073370 27519 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 208. Preceding OpId from leader: term: 1 index: 209. (index mismatch)
I20240502 14:00:18.073438 27254 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 208. Preceding OpId from leader: term: 1 index: 209. (index mismatch)
I20240502 14:00:18.074018 27381 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 208. Preceding OpId from leader: term: 1 index: 209. (index mismatch)
I20240502 14:00:18.074266 30167 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 209, Last known committed idx: 208, Time since last communication: 0.000s
I20240502 14:00:18.074595 30165 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 209, Last known committed idx: 208, Time since last communication: 0.000s
I20240502 14:00:18.074775 30166 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 209, Last known committed idx: 208, Time since last communication: 0.000s
I20240502 14:00:18.074944 28830 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 209, Last known committed idx: 208, Time since last communication: 0.000s
I20240502 14:00:18.077248 30165 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [term 1 LEADER]: Committing config change with OpId 1.209: config changed from index 67 to 209, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 209 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
W20240502 14:00:18.078425 27059 consensus_peers.cc:485] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a -> Peer b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72:46715): Couldn't send request to peer b15496b8de1b4bc084cdfbb7c581f361. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4a74c0c312a7403d89958a3632cab7d3. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:18.078502 27254 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.209: config changed from index 67 to 209, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 209 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:18.079514 26985 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.209: config changed from index 67 to 209, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 209 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:18.079846 27519 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.209: config changed from index 67 to 209, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 209 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:18.080500 27381 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.209: config changed from index 67 to 209, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 209 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:18.081171 26712 catalog_manager.cc:5484] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52 reported cstate change: config changed from index 67 to 209, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "0280081e4ff74896832ec3985c372b3a" committed_config { opid_index: 209 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:18.130350 29705 raft_consensus.cc:1050] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217: attempting to promote NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 to VOTER
I20240502 14:00:18.130741 29705 consensus_queue.cc:237] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 152, Committed index: 152, Last appended: 1.152, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } }
I20240502 14:00:18.132404 27117 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 152. Preceding OpId from leader: term: 1 index: 153. (index mismatch)
I20240502 14:00:18.132505 28365 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 152. Preceding OpId from leader: term: 1 index: 153. (index mismatch)
I20240502 14:00:18.132505 29608 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 152. Preceding OpId from leader: term: 1 index: 153. (index mismatch)
I20240502 14:00:18.132521 26858 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 152. Preceding OpId from leader: term: 1 index: 153. (index mismatch)
I20240502 14:00:18.132963 27519 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 152. Preceding OpId from leader: term: 1 index: 153. (index mismatch)
I20240502 14:00:18.132949 29705 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 153, Last known committed idx: 152, Time since last communication: 0.000s
I20240502 14:00:18.133139 30199 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 153, Last known committed idx: 152, Time since last communication: 0.000s
I20240502 14:00:18.133392 29705 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 153, Last known committed idx: 152, Time since last communication: 0.000s
I20240502 14:00:18.133530 30115 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 153, Last known committed idx: 152, Time since last communication: 0.000s
I20240502 14:00:18.133703 30199 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 153, Last known committed idx: 152, Time since last communication: 0.000s
I20240502 14:00:18.135937 30199 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [term 1 LEADER]: Committing config change with OpId 1.153: config changed from index 152 to 153, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:18.136420 26858 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.153: config changed from index 152 to 153, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:18.137579 27519 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.153: config changed from index 152 to 153, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:18.138406 27117 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.153: config changed from index 152 to 153, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:18.138551 29608 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Committing config change with OpId 1.153: config changed from index 152 to 153, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:18.140125 26709 catalog_manager.cc:5484] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68 reported cstate change: config changed from index 152 to 153, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e44f8c16a14a48a38a5a74af2899c217" committed_config { opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:18.141067 28365 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.153: config changed from index 152 to 153, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 153 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } attrs { replace: true } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:18.146031 27381 consensus_queue.cc:237] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 153, Committed index: 153, Last appended: 1.153, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 154 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } }
I20240502 14:00:18.147394 29608 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 153. Preceding OpId from leader: term: 1 index: 154. (index mismatch)
I20240502 14:00:18.147770 27519 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 153. Preceding OpId from leader: term: 1 index: 154. (index mismatch)
I20240502 14:00:18.147878 28365 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 153. Preceding OpId from leader: term: 1 index: 154. (index mismatch)
I20240502 14:00:18.147822 30199 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 154, Last known committed idx: 153, Time since last communication: 0.000s
I20240502 14:00:18.148212 30115 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 154, Last known committed idx: 153, Time since last communication: 0.000s
I20240502 14:00:18.148314 27117 raft_consensus.cc:1261] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 153. Preceding OpId from leader: term: 1 index: 154. (index mismatch)
I20240502 14:00:18.148545 30115 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 154, Last known committed idx: 153, Time since last communication: 0.000s
I20240502 14:00:18.148718 29953 consensus_queue.cc:1035] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 154, Last known committed idx: 153, Time since last communication: 0.000s
I20240502 14:00:18.150270 30115 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P e44f8c16a14a48a38a5a74af2899c217 [term 1 LEADER]: Committing config change with OpId 1.154: config changed from index 153 to 154, VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) evicted. New config: { opid_index: 154 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:18.150808 27117 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.154: config changed from index 153 to 154, VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) evicted. New config: { opid_index: 154 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:18.150887 28365 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.154: config changed from index 153 to 154, VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) evicted. New config: { opid_index: 154 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:18.152755 26701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 945d054a56ed4043b7b49d14ed917532 with cas_config_opid_index 153: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:18.153175 27519 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.154: config changed from index 153 to 154, VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) evicted. New config: { opid_index: 154 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:18.153484 26708 catalog_manager.cc:5484] T 945d054a56ed4043b7b49d14ed917532 P 0280081e4ff74896832ec3985c372b3a reported cstate change: config changed from index 153 to 154, VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) evicted. New cstate: current_term: 1 leader_uuid: "e44f8c16a14a48a38a5a74af2899c217" committed_config { opid_index: 154 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:18.153824 29608 raft_consensus.cc:2932] T 945d054a56ed4043b7b49d14ed917532 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Committing config change with OpId 1.154: config changed from index 153 to 154, VOTER d3e36186faa442b7bd87357e74d47f68 (127.24.66.65) evicted. New config: { opid_index: 154 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:18.157018 26837 tablet_service.cc:1514] Processing DeleteTablet for tablet 945d054a56ed4043b7b49d14ed917532 with delete_type TABLET_DATA_TOMBSTONED (TS d3e36186faa442b7bd87357e74d47f68 not found in new config with opid_index 154) from {username='slave'} at 127.0.0.1:45678
I20240502 14:00:18.157474 30205 tablet_replica.cc:330] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68: stopping tablet replica
I20240502 14:00:18.157693 30205 raft_consensus.cc:2229] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:18.157946 30205 raft_consensus.cc:2258] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:18.159544 30205 ts_tablet_manager.cc:1875] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:18.161625 30205 ts_tablet_manager.cc:1888] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.153
I20240502 14:00:18.161689 30205 log.cc:1196] T 945d054a56ed4043b7b49d14ed917532 P d3e36186faa442b7bd87357e74d47f68: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/wals/945d054a56ed4043b7b49d14ed917532
I20240502 14:00:18.162132 26700 catalog_manager.cc:4830] TS d3e36186faa442b7bd87357e74d47f68 (127.24.66.65:40841): tablet 945d054a56ed4043b7b49d14ed917532 (table rebalance_test_table_2 [id=72ce8466a12e4262b615aa4cc05d3b93]) successfully deleted
I20240502 14:00:18.236166 30206 ts_tablet_manager.cc:902] T f4bb31e5c0454e54a7b3855445120783 P b15496b8de1b4bc084cdfbb7c581f361: Initiating tablet copy from peer b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68:35167)
I20240502 14:00:18.236761 30206 tablet_copy_client.cc:308] T f4bb31e5c0454e54a7b3855445120783 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:35167
I20240502 14:00:18.237234 27275 tablet_copy_service.cc:140] P b4fc3c795b1d4505b3a2903532d38b52: Received BeginTabletCopySession request for tablet f4bb31e5c0454e54a7b3855445120783 from peer b15496b8de1b4bc084cdfbb7c581f361 ({username='slave'} at 127.24.66.72:51597)
I20240502 14:00:18.237360 27275 tablet_copy_service.cc:161] P b4fc3c795b1d4505b3a2903532d38b52: Beginning new tablet copy session on tablet f4bb31e5c0454e54a7b3855445120783 from peer b15496b8de1b4bc084cdfbb7c581f361 at {username='slave'} at 127.24.66.72:51597: session id = b15496b8de1b4bc084cdfbb7c581f361-f4bb31e5c0454e54a7b3855445120783
I20240502 14:00:18.238090 27275 tablet_copy_source_session.cc:205] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:18.238925 30206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4bb31e5c0454e54a7b3855445120783. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:18.240926 30206 tablet_copy_client.cc:783] T f4bb31e5c0454e54a7b3855445120783 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:18.241080 30206 tablet_copy_client.cc:647] T f4bb31e5c0454e54a7b3855445120783 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:18.243010 30206 tablet_copy_client.cc:523] T f4bb31e5c0454e54a7b3855445120783 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:18.244396 30206 tablet_bootstrap.cc:492] T f4bb31e5c0454e54a7b3855445120783 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap starting.
I20240502 14:00:18.249224 30211 ts_tablet_manager.cc:902] T 9efaa3c68c534606bd37bbe4703b2ab2 P b15496b8de1b4bc084cdfbb7c581f361: Initiating tablet copy from peer 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217)
I20240502 14:00:18.249742 30211 tablet_copy_client.cc:308] T 9efaa3c68c534606bd37bbe4703b2ab2 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.70:38217
I20240502 14:00:18.253937 27540 tablet_copy_service.cc:140] P 34b161bd719d4d72baf8da4f40aef136: Received BeginTabletCopySession request for tablet 9efaa3c68c534606bd37bbe4703b2ab2 from peer b15496b8de1b4bc084cdfbb7c581f361 ({username='slave'} at 127.24.66.72:53941)
I20240502 14:00:18.254077 27540 tablet_copy_service.cc:161] P 34b161bd719d4d72baf8da4f40aef136: Beginning new tablet copy session on tablet 9efaa3c68c534606bd37bbe4703b2ab2 from peer b15496b8de1b4bc084cdfbb7c581f361 at {username='slave'} at 127.24.66.72:53941: session id = b15496b8de1b4bc084cdfbb7c581f361-9efaa3c68c534606bd37bbe4703b2ab2
I20240502 14:00:18.255128 27540 tablet_copy_source_session.cc:205] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:18.256120 30211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9efaa3c68c534606bd37bbe4703b2ab2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:18.258677 30211 tablet_copy_client.cc:783] T 9efaa3c68c534606bd37bbe4703b2ab2 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:18.258865 30211 tablet_copy_client.cc:647] T 9efaa3c68c534606bd37bbe4703b2ab2 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:18.260919 30211 tablet_copy_client.cc:523] T 9efaa3c68c534606bd37bbe4703b2ab2 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:18.262326 30211 tablet_bootstrap.cc:492] T 9efaa3c68c534606bd37bbe4703b2ab2 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap starting.
I20240502 14:00:18.371281 27381 consensus_queue.cc:237] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 267, Committed index: 267, Last appended: 1.267, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 268 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:18.373503 26858 raft_consensus.cc:1261] T d6fcbe92870d40a6b4a860f5f4f81c29 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 267. Preceding OpId from leader: term: 1 index: 268. (index mismatch)
I20240502 14:00:18.373790 26985 raft_consensus.cc:1261] T d6fcbe92870d40a6b4a860f5f4f81c29 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 267. Preceding OpId from leader: term: 1 index: 268. (index mismatch)
I20240502 14:00:18.374166 27519 raft_consensus.cc:1261] T d6fcbe92870d40a6b4a860f5f4f81c29 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 267. Preceding OpId from leader: term: 1 index: 268. (index mismatch)
I20240502 14:00:18.374393 30116 consensus_queue.cc:1035] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 268, Last known committed idx: 267, Time since last communication: 0.000s
I20240502 14:00:18.375861 27254 raft_consensus.cc:1261] T d6fcbe92870d40a6b4a860f5f4f81c29 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 267. Preceding OpId from leader: term: 1 index: 268. (index mismatch)
W20240502 14:00:18.376206 27323 consensus_peers.cc:485] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 -> Peer b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72:46715): Couldn't send request to peer b15496b8de1b4bc084cdfbb7c581f361. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d6fcbe92870d40a6b4a860f5f4f81c29. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:18.377209 30116 consensus_queue.cc:1035] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 268, Last known committed idx: 267, Time since last communication: 0.000s
I20240502 14:00:18.378767 29705 consensus_queue.cc:1035] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 268, Last known committed idx: 267, Time since last communication: 0.000s
I20240502 14:00:18.379081 29705 consensus_queue.cc:1035] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 268, Last known committed idx: 267, Time since last communication: 0.000s
I20240502 14:00:18.382611 30116 raft_consensus.cc:2932] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [term 1 LEADER]: Committing config change with OpId 1.268: config changed from index -1 to 268, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 268 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:18.386297 26712 catalog_manager.cc:5484] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 reported cstate change: config changed from index -1 to 268, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "e44f8c16a14a48a38a5a74af2899c217" committed_config { opid_index: 268 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:18.387099 26985 raft_consensus.cc:2932] T d6fcbe92870d40a6b4a860f5f4f81c29 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.268: config changed from index -1 to 268, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 268 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:18.391768 27254 raft_consensus.cc:2932] T d6fcbe92870d40a6b4a860f5f4f81c29 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.268: config changed from index -1 to 268, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 268 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:18.391929 26858 raft_consensus.cc:2932] T d6fcbe92870d40a6b4a860f5f4f81c29 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.268: config changed from index -1 to 268, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 268 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:18.397099 27519 raft_consensus.cc:2932] T d6fcbe92870d40a6b4a860f5f4f81c29 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.268: config changed from index -1 to 268, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 268 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:18.407132 26858 consensus_queue.cc:237] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 294, Committed index: 294, Last appended: 1.294, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 295 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } }
I20240502 14:00:18.409045 29608 raft_consensus.cc:1261] T 53a68b75cbce41d69432f21f2cfbf256 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Refusing update from remote peer d3e36186faa442b7bd87357e74d47f68: Log matching property violated. Preceding OpId in replica: term: 1 index: 294. Preceding OpId from leader: term: 1 index: 295. (index mismatch)
I20240502 14:00:18.410842 27117 raft_consensus.cc:1261] T 53a68b75cbce41d69432f21f2cfbf256 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer d3e36186faa442b7bd87357e74d47f68: Log matching property violated. Preceding OpId in replica: term: 1 index: 294. Preceding OpId from leader: term: 1 index: 295. (index mismatch)
I20240502 14:00:18.411438 30126 consensus_queue.cc:1035] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 295, Last known committed idx: 294, Time since last communication: 0.000s
I20240502 14:00:18.411832 30126 consensus_queue.cc:1035] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 295, Last known committed idx: 294, Time since last communication: 0.000s
I20240502 14:00:18.411886 27519 raft_consensus.cc:1261] T 53a68b75cbce41d69432f21f2cfbf256 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer d3e36186faa442b7bd87357e74d47f68: Log matching property violated. Preceding OpId in replica: term: 1 index: 294. Preceding OpId from leader: term: 1 index: 295. (index mismatch)
I20240502 14:00:18.412315 29803 consensus_queue.cc:1035] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 295, Last known committed idx: 294, Time since last communication: 0.000s
I20240502 14:00:18.412410 27254 raft_consensus.cc:1261] T 53a68b75cbce41d69432f21f2cfbf256 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer d3e36186faa442b7bd87357e74d47f68: Log matching property violated. Preceding OpId in replica: term: 1 index: 294. Preceding OpId from leader: term: 1 index: 295. (index mismatch)
I20240502 14:00:18.415230 29803 consensus_queue.cc:1035] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 295, Last known committed idx: 294, Time since last communication: 0.000s
I20240502 14:00:18.416235 29803 raft_consensus.cc:2932] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 [term 1 LEADER]: Committing config change with OpId 1.295: config changed from index 294 to 295, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New config: { opid_index: 295 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:18.418807 26700 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 53a68b75cbce41d69432f21f2cfbf256 with cas_config_opid_index 294: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:18.419728 26712 catalog_manager.cc:5484] T 53a68b75cbce41d69432f21f2cfbf256 P d3e36186faa442b7bd87357e74d47f68 reported cstate change: config changed from index 294 to 295, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New cstate: current_term: 1 leader_uuid: "d3e36186faa442b7bd87357e74d47f68" committed_config { opid_index: 295 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:18.424947 27519 raft_consensus.cc:2932] T 53a68b75cbce41d69432f21f2cfbf256 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.295: config changed from index 294 to 295, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New config: { opid_index: 295 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:18.424963 29608 raft_consensus.cc:2932] T 53a68b75cbce41d69432f21f2cfbf256 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Committing config change with OpId 1.295: config changed from index 294 to 295, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New config: { opid_index: 295 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:18.428875 27254 raft_consensus.cc:2932] T 53a68b75cbce41d69432f21f2cfbf256 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.295: config changed from index 294 to 295, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New config: { opid_index: 295 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:18.431416 27117 raft_consensus.cc:2932] T 53a68b75cbce41d69432f21f2cfbf256 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.295: config changed from index 294 to 295, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New config: { opid_index: 295 OBSOLETE_local: false peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:18.436457 26971 tablet_service.cc:1514] Processing DeleteTablet for tablet 53a68b75cbce41d69432f21f2cfbf256 with delete_type TABLET_DATA_TOMBSTONED (TS 256ce64303ec49e1a5cd8e0ffee436b0 not found in new config with opid_index 295) from {username='slave'} at 127.0.0.1:51832
I20240502 14:00:18.439039 30264 tablet_replica.cc:330] T 53a68b75cbce41d69432f21f2cfbf256 P 256ce64303ec49e1a5cd8e0ffee436b0: stopping tablet replica
I20240502 14:00:18.439239 30264 raft_consensus.cc:2229] T 53a68b75cbce41d69432f21f2cfbf256 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:18.439626 30264 raft_consensus.cc:2258] T 53a68b75cbce41d69432f21f2cfbf256 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:18.443269 30264 ts_tablet_manager.cc:1875] T 53a68b75cbce41d69432f21f2cfbf256 P 256ce64303ec49e1a5cd8e0ffee436b0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:18.446734 30264 ts_tablet_manager.cc:1888] T 53a68b75cbce41d69432f21f2cfbf256 P 256ce64303ec49e1a5cd8e0ffee436b0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.294
I20240502 14:00:18.446883 30264 log.cc:1196] T 53a68b75cbce41d69432f21f2cfbf256 P 256ce64303ec49e1a5cd8e0ffee436b0: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/wals/53a68b75cbce41d69432f21f2cfbf256
I20240502 14:00:18.447878 26700 catalog_manager.cc:4830] TS 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473): tablet 53a68b75cbce41d69432f21f2cfbf256 (table rebalance_test_table_0 [id=360c53214ba44103b4436952fc42eddc]) successfully deleted
I20240502 14:00:18.466333 30206 tablet_bootstrap.cc:492] T f4bb31e5c0454e54a7b3855445120783 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap replayed 1/1 log segments. Stats: ops{read=319 overwritten=0 applied=319 ignored=0} inserts{seen=1001 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:18.466709 30206 tablet_bootstrap.cc:492] T f4bb31e5c0454e54a7b3855445120783 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap complete.
I20240502 14:00:18.467278 30206 ts_tablet_manager.cc:1367] T f4bb31e5c0454e54a7b3855445120783 P b15496b8de1b4bc084cdfbb7c581f361: Time spent bootstrapping tablet: real 0.223s user 0.197s sys 0.024s
I20240502 14:00:18.467762 30206 raft_consensus.cc:348] T f4bb31e5c0454e54a7b3855445120783 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 319 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false replace: true } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:18.467907 30206 raft_consensus.cc:729] T f4bb31e5c0454e54a7b3855445120783 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b15496b8de1b4bc084cdfbb7c581f361, State: Initialized, Role: LEARNER
I20240502 14:00:18.468024 30206 consensus_queue.cc:260] T f4bb31e5c0454e54a7b3855445120783 P b15496b8de1b4bc084cdfbb7c581f361 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 319, Last appended: 1.319, Last appended by leader: 319, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 319 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false replace: true } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:18.468307 30206 ts_tablet_manager.cc:1398] T f4bb31e5c0454e54a7b3855445120783 P b15496b8de1b4bc084cdfbb7c581f361: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:18.468894 27275 tablet_copy_service.cc:342] P b4fc3c795b1d4505b3a2903532d38b52: Request end of tablet copy session b15496b8de1b4bc084cdfbb7c581f361-f4bb31e5c0454e54a7b3855445120783 received from {username='slave'} at 127.24.66.72:51597
I20240502 14:00:18.468991 27275 tablet_copy_service.cc:433] P b4fc3c795b1d4505b3a2903532d38b52: ending tablet copy session b15496b8de1b4bc084cdfbb7c581f361-f4bb31e5c0454e54a7b3855445120783 on tablet f4bb31e5c0454e54a7b3855445120783 with peer b15496b8de1b4bc084cdfbb7c581f361
I20240502 14:00:18.473192 27381 consensus_queue.cc:237] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:18.474895 28365 raft_consensus.cc:1261] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:18.475299 27254 raft_consensus.cc:1261] T 4a156115301a4d40bfc7f2bd11f135bf P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:18.475497 30115 consensus_queue.cc:1035] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:18.475842 27519 raft_consensus.cc:1261] T 4a156115301a4d40bfc7f2bd11f135bf P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:18.476311 29705 consensus_queue.cc:1035] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 14:00:18.476552 27323 consensus_peers.cc:485] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 -> Peer b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72:46715): Couldn't send request to peer b15496b8de1b4bc084cdfbb7c581f361. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4a156115301a4d40bfc7f2bd11f135bf. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:18.476750 29705 consensus_queue.cc:1035] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:18.477842 26985 raft_consensus.cc:1261] T 4a156115301a4d40bfc7f2bd11f135bf P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:18.478883 29953 consensus_queue.cc:1035] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 14:00:18.479408 30115 raft_consensus.cc:2932] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:18.480739 27254 raft_consensus.cc:2932] T 4a156115301a4d40bfc7f2bd11f135bf P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:18.482380 28365 raft_consensus.cc:2932] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:18.483412 26714 catalog_manager.cc:5484] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 reported cstate change: config changed from index -1 to 2, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "e44f8c16a14a48a38a5a74af2899c217" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:18.484867 27519 raft_consensus.cc:2932] T 4a156115301a4d40bfc7f2bd11f135bf P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:18.487928 26985 raft_consensus.cc:2932] T 4a156115301a4d40bfc7f2bd11f135bf P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } } }
I20240502 14:00:18.496207 30211 tablet_bootstrap.cc:492] T 9efaa3c68c534606bd37bbe4703b2ab2 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap replayed 1/1 log segments. Stats: ops{read=289 overwritten=0 applied=289 ignored=0} inserts{seen=971 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:18.496551 30211 tablet_bootstrap.cc:492] T 9efaa3c68c534606bd37bbe4703b2ab2 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap complete.
I20240502 14:00:18.496745 30211 ts_tablet_manager.cc:1367] T 9efaa3c68c534606bd37bbe4703b2ab2 P b15496b8de1b4bc084cdfbb7c581f361: Time spent bootstrapping tablet: real 0.234s user 0.198s sys 0.020s
I20240502 14:00:18.497144 30211 raft_consensus.cc:348] T 9efaa3c68c534606bd37bbe4703b2ab2 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 289 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:18.497248 30211 raft_consensus.cc:729] T 9efaa3c68c534606bd37bbe4703b2ab2 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b15496b8de1b4bc084cdfbb7c581f361, State: Initialized, Role: LEARNER
I20240502 14:00:18.497357 30211 consensus_queue.cc:260] T 9efaa3c68c534606bd37bbe4703b2ab2 P b15496b8de1b4bc084cdfbb7c581f361 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 289, Last appended: 1.289, Last appended by leader: 289, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 289 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:18.497663 30211 ts_tablet_manager.cc:1398] T 9efaa3c68c534606bd37bbe4703b2ab2 P b15496b8de1b4bc084cdfbb7c581f361: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:18.498287 27540 tablet_copy_service.cc:342] P 34b161bd719d4d72baf8da4f40aef136: Request end of tablet copy session b15496b8de1b4bc084cdfbb7c581f361-9efaa3c68c534606bd37bbe4703b2ab2 received from {username='slave'} at 127.24.66.72:53941
I20240502 14:00:18.498384 27540 tablet_copy_service.cc:433] P 34b161bd719d4d72baf8da4f40aef136: ending tablet copy session b15496b8de1b4bc084cdfbb7c581f361-9efaa3c68c534606bd37bbe4703b2ab2 on tablet 9efaa3c68c534606bd37bbe4703b2ab2 with peer b15496b8de1b4bc084cdfbb7c581f361
I20240502 14:00:18.580183 30211 ts_tablet_manager.cc:902] T 4a74c0c312a7403d89958a3632cab7d3 P b15496b8de1b4bc084cdfbb7c581f361: Initiating tablet copy from peer 0280081e4ff74896832ec3985c372b3a (127.24.66.67:35733)
I20240502 14:00:18.580679 30211 tablet_copy_client.cc:308] T 4a74c0c312a7403d89958a3632cab7d3 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.67:35733
I20240502 14:00:18.583878 27144 tablet_copy_service.cc:140] P 0280081e4ff74896832ec3985c372b3a: Received BeginTabletCopySession request for tablet 4a74c0c312a7403d89958a3632cab7d3 from peer b15496b8de1b4bc084cdfbb7c581f361 ({username='slave'} at 127.24.66.72:33713)
I20240502 14:00:18.583969 27144 tablet_copy_service.cc:161] P 0280081e4ff74896832ec3985c372b3a: Beginning new tablet copy session on tablet 4a74c0c312a7403d89958a3632cab7d3 from peer b15496b8de1b4bc084cdfbb7c581f361 at {username='slave'} at 127.24.66.72:33713: session id = b15496b8de1b4bc084cdfbb7c581f361-4a74c0c312a7403d89958a3632cab7d3
I20240502 14:00:18.584738 27144 tablet_copy_source_session.cc:205] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:18.585505 30211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a74c0c312a7403d89958a3632cab7d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:18.587141 30211 tablet_copy_client.cc:783] T 4a74c0c312a7403d89958a3632cab7d3 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:18.587294 30211 tablet_copy_client.cc:647] T 4a74c0c312a7403d89958a3632cab7d3 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:18.588553 30211 tablet_copy_client.cc:523] T 4a74c0c312a7403d89958a3632cab7d3 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:18.589453 30211 tablet_bootstrap.cc:492] T 4a74c0c312a7403d89958a3632cab7d3 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap starting.
I20240502 14:00:18.716814 30211 tablet_bootstrap.cc:492] T 4a74c0c312a7403d89958a3632cab7d3 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap replayed 1/1 log segments. Stats: ops{read=209 overwritten=0 applied=209 ignored=0} inserts{seen=891 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:18.717132 30211 tablet_bootstrap.cc:492] T 4a74c0c312a7403d89958a3632cab7d3 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap complete.
I20240502 14:00:18.717350 30211 ts_tablet_manager.cc:1367] T 4a74c0c312a7403d89958a3632cab7d3 P b15496b8de1b4bc084cdfbb7c581f361: Time spent bootstrapping tablet: real 0.128s user 0.123s sys 0.004s
I20240502 14:00:18.717770 30211 raft_consensus.cc:348] T 4a74c0c312a7403d89958a3632cab7d3 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 209 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:18.717892 30211 raft_consensus.cc:729] T 4a74c0c312a7403d89958a3632cab7d3 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b15496b8de1b4bc084cdfbb7c581f361, State: Initialized, Role: LEARNER
I20240502 14:00:18.718001 30211 consensus_queue.cc:260] T 4a74c0c312a7403d89958a3632cab7d3 P b15496b8de1b4bc084cdfbb7c581f361 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 209, Last appended: 1.209, Last appended by leader: 209, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 209 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:18.718283 30211 ts_tablet_manager.cc:1398] T 4a74c0c312a7403d89958a3632cab7d3 P b15496b8de1b4bc084cdfbb7c581f361: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:18.718948 27144 tablet_copy_service.cc:342] P 0280081e4ff74896832ec3985c372b3a: Request end of tablet copy session b15496b8de1b4bc084cdfbb7c581f361-4a74c0c312a7403d89958a3632cab7d3 received from {username='slave'} at 127.24.66.72:33713
I20240502 14:00:18.719048 27144 tablet_copy_service.cc:433] P 0280081e4ff74896832ec3985c372b3a: ending tablet copy session b15496b8de1b4bc084cdfbb7c581f361-4a74c0c312a7403d89958a3632cab7d3 on tablet 4a74c0c312a7403d89958a3632cab7d3 with peer b15496b8de1b4bc084cdfbb7c581f361
I20240502 14:00:18.730137 29608 raft_consensus.cc:1203] T 9efaa3c68c534606bd37bbe4703b2ab2 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.288->[1.289-1.289] Dedup: 1.289->[]
I20240502 14:00:18.757722 30211 ts_tablet_manager.cc:902] T d6fcbe92870d40a6b4a860f5f4f81c29 P b15496b8de1b4bc084cdfbb7c581f361: Initiating tablet copy from peer e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107)
I20240502 14:00:18.758155 30211 tablet_copy_client.cc:308] T d6fcbe92870d40a6b4a860f5f4f81c29 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.69:39107
I20240502 14:00:18.758603 27408 tablet_copy_service.cc:140] P e44f8c16a14a48a38a5a74af2899c217: Received BeginTabletCopySession request for tablet d6fcbe92870d40a6b4a860f5f4f81c29 from peer b15496b8de1b4bc084cdfbb7c581f361 ({username='slave'} at 127.24.66.72:32821)
I20240502 14:00:18.758705 27408 tablet_copy_service.cc:161] P e44f8c16a14a48a38a5a74af2899c217: Beginning new tablet copy session on tablet d6fcbe92870d40a6b4a860f5f4f81c29 from peer b15496b8de1b4bc084cdfbb7c581f361 at {username='slave'} at 127.24.66.72:32821: session id = b15496b8de1b4bc084cdfbb7c581f361-d6fcbe92870d40a6b4a860f5f4f81c29
I20240502 14:00:18.759467 27408 tablet_copy_source_session.cc:205] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:18.760233 30211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6fcbe92870d40a6b4a860f5f4f81c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:18.761934 30211 tablet_copy_client.cc:783] T d6fcbe92870d40a6b4a860f5f4f81c29 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:18.762097 30211 tablet_copy_client.cc:647] T d6fcbe92870d40a6b4a860f5f4f81c29 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:18.763629 30211 tablet_copy_client.cc:523] T d6fcbe92870d40a6b4a860f5f4f81c29 P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:18.764636 30211 tablet_bootstrap.cc:492] T d6fcbe92870d40a6b4a860f5f4f81c29 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap starting.
I20240502 14:00:18.837109 29608 raft_consensus.cc:1203] T f4bb31e5c0454e54a7b3855445120783 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.318->[1.319-1.319] Dedup: 1.319->[]
I20240502 14:00:18.927423 30211 tablet_bootstrap.cc:492] T d6fcbe92870d40a6b4a860f5f4f81c29 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap replayed 1/1 log segments. Stats: ops{read=268 overwritten=0 applied=268 ignored=0} inserts{seen=940 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:18.927706 30211 tablet_bootstrap.cc:492] T d6fcbe92870d40a6b4a860f5f4f81c29 P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap complete.
I20240502 14:00:18.927897 30211 ts_tablet_manager.cc:1367] T d6fcbe92870d40a6b4a860f5f4f81c29 P b15496b8de1b4bc084cdfbb7c581f361: Time spent bootstrapping tablet: real 0.163s user 0.156s sys 0.004s
I20240502 14:00:18.928296 30211 raft_consensus.cc:348] T d6fcbe92870d40a6b4a860f5f4f81c29 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 268 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:18.928452 30211 raft_consensus.cc:729] T d6fcbe92870d40a6b4a860f5f4f81c29 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b15496b8de1b4bc084cdfbb7c581f361, State: Initialized, Role: LEARNER
I20240502 14:00:18.928562 30211 consensus_queue.cc:260] T d6fcbe92870d40a6b4a860f5f4f81c29 P b15496b8de1b4bc084cdfbb7c581f361 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 268, Last appended: 1.268, Last appended by leader: 268, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 268 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:18.928853 30211 ts_tablet_manager.cc:1398] T d6fcbe92870d40a6b4a860f5f4f81c29 P b15496b8de1b4bc084cdfbb7c581f361: Time spent starting tablet: real 0.001s user 0.000s sys 0.004s
I20240502 14:00:18.929371 27408 tablet_copy_service.cc:342] P e44f8c16a14a48a38a5a74af2899c217: Request end of tablet copy session b15496b8de1b4bc084cdfbb7c581f361-d6fcbe92870d40a6b4a860f5f4f81c29 received from {username='slave'} at 127.24.66.72:32821
I20240502 14:00:18.929467 27408 tablet_copy_service.cc:433] P e44f8c16a14a48a38a5a74af2899c217: ending tablet copy session b15496b8de1b4bc084cdfbb7c581f361-d6fcbe92870d40a6b4a860f5f4f81c29 on tablet d6fcbe92870d40a6b4a860f5f4f81c29 with peer b15496b8de1b4bc084cdfbb7c581f361
I20240502 14:00:18.937059 30211 ts_tablet_manager.cc:902] T 4a156115301a4d40bfc7f2bd11f135bf P b15496b8de1b4bc084cdfbb7c581f361: Initiating tablet copy from peer e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107)
I20240502 14:00:18.937434 30211 tablet_copy_client.cc:308] T 4a156115301a4d40bfc7f2bd11f135bf P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.69:39107
I20240502 14:00:18.937846 27408 tablet_copy_service.cc:140] P e44f8c16a14a48a38a5a74af2899c217: Received BeginTabletCopySession request for tablet 4a156115301a4d40bfc7f2bd11f135bf from peer b15496b8de1b4bc084cdfbb7c581f361 ({username='slave'} at 127.24.66.72:32821)
I20240502 14:00:18.937942 27408 tablet_copy_service.cc:161] P e44f8c16a14a48a38a5a74af2899c217: Beginning new tablet copy session on tablet 4a156115301a4d40bfc7f2bd11f135bf from peer b15496b8de1b4bc084cdfbb7c581f361 at {username='slave'} at 127.24.66.72:32821: session id = b15496b8de1b4bc084cdfbb7c581f361-4a156115301a4d40bfc7f2bd11f135bf
I20240502 14:00:18.938678 27408 tablet_copy_source_session.cc:205] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:18.939384 30211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a156115301a4d40bfc7f2bd11f135bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:18.941094 30211 tablet_copy_client.cc:783] T 4a156115301a4d40bfc7f2bd11f135bf P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:18.941237 30211 tablet_copy_client.cc:647] T 4a156115301a4d40bfc7f2bd11f135bf P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:18.942296 30211 tablet_copy_client.cc:523] T 4a156115301a4d40bfc7f2bd11f135bf P b15496b8de1b4bc084cdfbb7c581f361: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:18.943269 30211 tablet_bootstrap.cc:492] T 4a156115301a4d40bfc7f2bd11f135bf P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap starting.
I20240502 14:00:18.946112 30211 tablet_bootstrap.cc:492] T 4a156115301a4d40bfc7f2bd11f135bf P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:18.946317 30211 tablet_bootstrap.cc:492] T 4a156115301a4d40bfc7f2bd11f135bf P b15496b8de1b4bc084cdfbb7c581f361: Bootstrap complete.
I20240502 14:00:18.946460 30211 ts_tablet_manager.cc:1367] T 4a156115301a4d40bfc7f2bd11f135bf P b15496b8de1b4bc084cdfbb7c581f361: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 14:00:18.946861 30211 raft_consensus.cc:348] T 4a156115301a4d40bfc7f2bd11f135bf P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:18.946992 30211 raft_consensus.cc:729] T 4a156115301a4d40bfc7f2bd11f135bf P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b15496b8de1b4bc084cdfbb7c581f361, State: Initialized, Role: LEARNER
I20240502 14:00:18.947108 30211 consensus_queue.cc:260] T 4a156115301a4d40bfc7f2bd11f135bf P b15496b8de1b4bc084cdfbb7c581f361 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: true } }
I20240502 14:00:18.947458 30211 ts_tablet_manager.cc:1398] T 4a156115301a4d40bfc7f2bd11f135bf P b15496b8de1b4bc084cdfbb7c581f361: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:00:18.948128 27408 tablet_copy_service.cc:342] P e44f8c16a14a48a38a5a74af2899c217: Request end of tablet copy session b15496b8de1b4bc084cdfbb7c581f361-4a156115301a4d40bfc7f2bd11f135bf received from {username='slave'} at 127.24.66.72:32821
I20240502 14:00:18.948196 27408 tablet_copy_service.cc:433] P e44f8c16a14a48a38a5a74af2899c217: ending tablet copy session b15496b8de1b4bc084cdfbb7c581f361-4a156115301a4d40bfc7f2bd11f135bf on tablet 4a156115301a4d40bfc7f2bd11f135bf with peer b15496b8de1b4bc084cdfbb7c581f361
I20240502 14:00:19.122644 29608 raft_consensus.cc:1203] T 4a74c0c312a7403d89958a3632cab7d3 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.208->[1.209-1.209] Dedup: 1.209->[]
I20240502 14:00:19.247136 30331 raft_consensus.cc:1050] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136: attempting to promote NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 to VOTER
I20240502 14:00:19.247572 30331 consensus_queue.cc:237] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 289, Committed index: 289, Last appended: 1.289, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 290 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } }
I20240502 14:00:19.248960 29608 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 289. Preceding OpId from leader: term: 1 index: 290. (index mismatch)
I20240502 14:00:19.249226 28363 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 289. Preceding OpId from leader: term: 1 index: 290. (index mismatch)
I20240502 14:00:19.249311 27381 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 289. Preceding OpId from leader: term: 1 index: 290. (index mismatch)
I20240502 14:00:19.249626 27117 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 289. Preceding OpId from leader: term: 1 index: 290. (index mismatch)
I20240502 14:00:19.249715 27254 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 289. Preceding OpId from leader: term: 1 index: 290. (index mismatch)
I20240502 14:00:19.249689 30412 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 290, Last known committed idx: 289, Time since last communication: 0.000s
I20240502 14:00:19.250082 30417 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 290, Last known committed idx: 289, Time since last communication: 0.000s
I20240502 14:00:19.250311 30414 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 290, Last known committed idx: 289, Time since last communication: 0.000s
I20240502 14:00:19.250438 30416 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 290, Last known committed idx: 289, Time since last communication: 0.000s
I20240502 14:00:19.250694 30415 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 290, Last known committed idx: 289, Time since last communication: 0.000s
I20240502 14:00:19.252979 30415 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [term 1 LEADER]: Committing config change with OpId 1.290: config changed from index 289 to 290, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 290 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.254128 28363 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.290: config changed from index 289 to 290, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 290 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.255563 27381 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.290: config changed from index 289 to 290, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 290 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.255700 27254 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.290: config changed from index 289 to 290, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 290 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.255761 27117 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.290: config changed from index 289 to 290, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 290 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.256103 26714 catalog_manager.cc:5484] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 reported cstate change: config changed from index 289 to 290, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "34b161bd719d4d72baf8da4f40aef136" committed_config { opid_index: 290 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:19.257839 29608 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Committing config change with OpId 1.290: config changed from index 289 to 290, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 290 OBSOLETE_local: false peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.266034 27519 consensus_queue.cc:237] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 290, Committed index: 290, Last appended: 1.290, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } }
I20240502 14:00:19.267483 28363 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 290. Preceding OpId from leader: term: 1 index: 291. (index mismatch)
I20240502 14:00:19.267638 27254 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 290. Preceding OpId from leader: term: 1 index: 291. (index mismatch)
I20240502 14:00:19.267835 29608 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 290. Preceding OpId from leader: term: 1 index: 291. (index mismatch)
I20240502 14:00:19.267885 30412 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 291, Last known committed idx: 290, Time since last communication: 0.000s
I20240502 14:00:19.268141 30415 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 291, Last known committed idx: 290, Time since last communication: 0.000s
I20240502 14:00:19.268188 27117 raft_consensus.cc:1261] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer 34b161bd719d4d72baf8da4f40aef136: Log matching property violated. Preceding OpId in replica: term: 1 index: 290. Preceding OpId from leader: term: 1 index: 291. (index mismatch)
I20240502 14:00:19.268282 30412 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 291, Last known committed idx: 290, Time since last communication: 0.000s
I20240502 14:00:19.268579 30416 consensus_queue.cc:1035] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 291, Last known committed idx: 290, Time since last communication: 0.000s
I20240502 14:00:19.270224 30412 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P 34b161bd719d4d72baf8da4f40aef136 [term 1 LEADER]: Committing config change with OpId 1.291: config changed from index 290 to 291, VOTER e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69) evicted. New config: { opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.270671 27117 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.291: config changed from index 290 to 291, VOTER e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69) evicted. New config: { opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.270876 28363 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.291: config changed from index 290 to 291, VOTER e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69) evicted. New config: { opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.272428 26700 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 9efaa3c68c534606bd37bbe4703b2ab2 with cas_config_opid_index 290: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:19.272915 27254 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.291: config changed from index 290 to 291, VOTER e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69) evicted. New config: { opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.273676 26712 catalog_manager.cc:5484] T 9efaa3c68c534606bd37bbe4703b2ab2 P 0280081e4ff74896832ec3985c372b3a reported cstate change: config changed from index 290 to 291, VOTER e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69) evicted. New cstate: current_term: 1 leader_uuid: "34b161bd719d4d72baf8da4f40aef136" committed_config { opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.274863 29608 raft_consensus.cc:2932] T 9efaa3c68c534606bd37bbe4703b2ab2 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Committing config change with OpId 1.291: config changed from index 290 to 291, VOTER e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69) evicted. New config: { opid_index: 291 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.276255 27367 tablet_service.cc:1514] Processing DeleteTablet for tablet 9efaa3c68c534606bd37bbe4703b2ab2 with delete_type TABLET_DATA_TOMBSTONED (TS e44f8c16a14a48a38a5a74af2899c217 not found in new config with opid_index 291) from {username='slave'} at 127.0.0.1:49794
I20240502 14:00:19.277659 30428 tablet_replica.cc:330] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217: stopping tablet replica
I20240502 14:00:19.277875 30428 raft_consensus.cc:2229] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:19.278069 30428 raft_consensus.cc:2258] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:19.279660 30428 ts_tablet_manager.cc:1875] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:19.281894 30428 ts_tablet_manager.cc:1888] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.290
I20240502 14:00:19.281965 30428 log.cc:1196] T 9efaa3c68c534606bd37bbe4703b2ab2 P e44f8c16a14a48a38a5a74af2899c217: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal/wals/9efaa3c68c534606bd37bbe4703b2ab2
I20240502 14:00:19.282531 26701 catalog_manager.cc:4830] TS e44f8c16a14a48a38a5a74af2899c217 (127.24.66.69:39107): tablet 9efaa3c68c534606bd37bbe4703b2ab2 (table rebalance_test_table_3 [id=ecda844ae2ee4cb5a179b79c8dacb5ac]) successfully deleted
I20240502 14:00:19.302031 29995 raft_consensus.cc:1050] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52: attempting to promote NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 to VOTER
I20240502 14:00:19.302488 29995 consensus_queue.cc:237] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 319, Committed index: 319, Last appended: 1.319, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 320 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false replace: true } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } }
I20240502 14:00:19.304111 27381 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 319. Preceding OpId from leader: term: 1 index: 320. (index mismatch)
I20240502 14:00:19.304111 27117 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 319. Preceding OpId from leader: term: 1 index: 320. (index mismatch)
I20240502 14:00:19.304147 28363 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 319. Preceding OpId from leader: term: 1 index: 320. (index mismatch)
I20240502 14:00:19.304312 29608 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 319. Preceding OpId from leader: term: 1 index: 320. (index mismatch)
I20240502 14:00:19.304575 29995 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 320, Last known committed idx: 319, Time since last communication: 0.000s
I20240502 14:00:19.304704 26985 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 319. Preceding OpId from leader: term: 1 index: 320. (index mismatch)
I20240502 14:00:19.304741 29750 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 320, Last known committed idx: 319, Time since last communication: 0.000s
I20240502 14:00:19.304927 30432 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 320, Last known committed idx: 319, Time since last communication: 0.000s
I20240502 14:00:19.305094 30431 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 320, Last known committed idx: 319, Time since last communication: 0.000s
I20240502 14:00:19.305235 30352 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 320, Last known committed idx: 319, Time since last communication: 0.000s
I20240502 14:00:19.307384 29750 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 LEADER]: Committing config change with OpId 1.320: config changed from index 319 to 320, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 320 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false replace: true } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.308197 28363 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.320: config changed from index 319 to 320, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 320 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false replace: true } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.308172 26985 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.320: config changed from index 319 to 320, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 320 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false replace: true } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.309985 27381 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.320: config changed from index 319 to 320, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 320 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false replace: true } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.309985 29608 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Committing config change with OpId 1.320: config changed from index 319 to 320, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 320 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false replace: true } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.310020 27117 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.320: config changed from index 319 to 320, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 320 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false replace: true } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.312090 26710 catalog_manager.cc:5484] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649 reported cstate change: config changed from index 319 to 320, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "b4fc3c795b1d4505b3a2903532d38b52" committed_config { opid_index: 320 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } attrs { promote: false replace: true } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.316064 27254 consensus_queue.cc:237] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 320, Committed index: 320, Last appended: 1.320, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 321 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } }
I20240502 14:00:19.317282 27117 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 320. Preceding OpId from leader: term: 1 index: 321. (index mismatch)
I20240502 14:00:19.317361 29608 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 320. Preceding OpId from leader: term: 1 index: 321. (index mismatch)
I20240502 14:00:19.317404 27381 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 320. Preceding OpId from leader: term: 1 index: 321. (index mismatch)
I20240502 14:00:19.317471 26985 raft_consensus.cc:1261] T f4bb31e5c0454e54a7b3855445120783 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer b4fc3c795b1d4505b3a2903532d38b52: Log matching property violated. Preceding OpId in replica: term: 1 index: 320. Preceding OpId from leader: term: 1 index: 321. (index mismatch)
I20240502 14:00:19.317787 30431 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 321, Last known committed idx: 320, Time since last communication: 0.000s
I20240502 14:00:19.317946 30430 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 321, Last known committed idx: 320, Time since last communication: 0.000s
I20240502 14:00:19.318094 29995 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 321, Last known committed idx: 320, Time since last communication: 0.000s
I20240502 14:00:19.318228 30432 consensus_queue.cc:1035] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 321, Last known committed idx: 320, Time since last communication: 0.000s
I20240502 14:00:19.319964 30432 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 LEADER]: Committing config change with OpId 1.321: config changed from index 320 to 321, VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) evicted. New config: { opid_index: 321 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.320534 27381 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.321: config changed from index 320 to 321, VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) evicted. New config: { opid_index: 321 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.321449 29608 raft_consensus.cc:1203] T d6fcbe92870d40a6b4a860f5f4f81c29 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.267->[1.268-1.268] Dedup: 1.268->[]
I20240502 14:00:19.321445 29609 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Committing config change with OpId 1.321: config changed from index 320 to 321, VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) evicted. New config: { opid_index: 321 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.322268 26703 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f4bb31e5c0454e54a7b3855445120783 with cas_config_opid_index 320: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:19.322970 27117 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P 0280081e4ff74896832ec3985c372b3a [term 1 FOLLOWER]: Committing config change with OpId 1.321: config changed from index 320 to 321, VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) evicted. New config: { opid_index: 321 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.324191 26985 raft_consensus.cc:2932] T f4bb31e5c0454e54a7b3855445120783 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.321: config changed from index 320 to 321, VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) evicted. New config: { opid_index: 321 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.325191 26713 catalog_manager.cc:5484] T f4bb31e5c0454e54a7b3855445120783 P b4fc3c795b1d4505b3a2903532d38b52 reported cstate change: config changed from index 320 to 321, VOTER 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71) evicted. New cstate: current_term: 1 leader_uuid: "b4fc3c795b1d4505b3a2903532d38b52" committed_config { opid_index: 321 OBSOLETE_local: false peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:19.331741 28345 tablet_service.cc:1514] Processing DeleteTablet for tablet f4bb31e5c0454e54a7b3855445120783 with delete_type TABLET_DATA_TOMBSTONED (TS 5f9ddfa1f6fc484d8fb96075a8efb649 not found in new config with opid_index 321) from {username='slave'} at 127.0.0.1:55322
I20240502 14:00:19.332242 30441 tablet_replica.cc:330] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649: stopping tablet replica
I20240502 14:00:19.332533 30441 raft_consensus.cc:2229] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:19.332772 30441 raft_consensus.cc:2258] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:19.334379 30441 ts_tablet_manager.cc:1875] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:19.336225 30441 ts_tablet_manager.cc:1888] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.320
I20240502 14:00:19.336297 30441 log.cc:1196] T f4bb31e5c0454e54a7b3855445120783 P 5f9ddfa1f6fc484d8fb96075a8efb649: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal/wals/f4bb31e5c0454e54a7b3855445120783
I20240502 14:00:19.336877 26700 catalog_manager.cc:4830] TS 5f9ddfa1f6fc484d8fb96075a8efb649 (127.24.66.71:42201): tablet f4bb31e5c0454e54a7b3855445120783 (table rebalance_test_table_0 [id=360c53214ba44103b4436952fc42eddc]) successfully deleted
I20240502 14:00:19.527516 29609 raft_consensus.cc:1203] T 4a156115301a4d40bfc7f2bd11f135bf P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 14:00:19.680972 30385 raft_consensus.cc:1050] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a: attempting to promote NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 to VOTER
I20240502 14:00:19.681411 30385 consensus_queue.cc:237] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 209, Committed index: 209, Last appended: 1.209, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 210 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } }
I20240502 14:00:19.682590 29609 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 209. Preceding OpId from leader: term: 1 index: 210. (index mismatch)
I20240502 14:00:19.683058 26985 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 209. Preceding OpId from leader: term: 1 index: 210. (index mismatch)
I20240502 14:00:19.683157 27381 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 209. Preceding OpId from leader: term: 1 index: 210. (index mismatch)
I20240502 14:00:19.683499 30502 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 210, Last known committed idx: 209, Time since last communication: 0.000s
I20240502 14:00:19.683555 27519 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 209. Preceding OpId from leader: term: 1 index: 210. (index mismatch)
I20240502 14:00:19.683707 27254 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 209. Preceding OpId from leader: term: 1 index: 210. (index mismatch)
I20240502 14:00:19.683845 30498 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 210, Last known committed idx: 209, Time since last communication: 0.000s
I20240502 14:00:19.684005 30502 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 210, Last known committed idx: 209, Time since last communication: 0.000s
I20240502 14:00:19.684163 30385 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 210, Last known committed idx: 209, Time since last communication: 0.000s
I20240502 14:00:19.684433 30385 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 210, Last known committed idx: 209, Time since last communication: 0.000s
I20240502 14:00:19.687124 30502 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [term 1 LEADER]: Committing config change with OpId 1.210: config changed from index 209 to 210, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 210 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.687544 27381 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.210: config changed from index 209 to 210, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 210 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.689471 27519 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.210: config changed from index 209 to 210, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 210 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.689945 29609 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Committing config change with OpId 1.210: config changed from index 209 to 210, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 210 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.690619 26709 catalog_manager.cc:5484] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a reported cstate change: config changed from index 209 to 210, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0280081e4ff74896832ec3985c372b3a" committed_config { opid_index: 210 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:19.690995 27254 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.210: config changed from index 209 to 210, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 210 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.693607 26985 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.210: config changed from index 209 to 210, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 210 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } attrs { replace: true } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.697084 27117 consensus_queue.cc:237] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 210, Committed index: 210, Last appended: 1.210, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 211 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } }
I20240502 14:00:19.698207 29609 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 210. Preceding OpId from leader: term: 1 index: 211. (index mismatch)
I20240502 14:00:19.698446 27519 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 210. Preceding OpId from leader: term: 1 index: 211. (index mismatch)
I20240502 14:00:19.698570 30500 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 211, Last known committed idx: 210, Time since last communication: 0.000s
I20240502 14:00:19.698765 26985 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 210. Preceding OpId from leader: term: 1 index: 211. (index mismatch)
I20240502 14:00:19.699026 30498 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 211, Last known committed idx: 210, Time since last communication: 0.000s
I20240502 14:00:19.699152 30501 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 211, Last known committed idx: 210, Time since last communication: 0.000s
I20240502 14:00:19.699950 27381 raft_consensus.cc:1261] T 4a74c0c312a7403d89958a3632cab7d3 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Refusing update from remote peer 0280081e4ff74896832ec3985c372b3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 210. Preceding OpId from leader: term: 1 index: 211. (index mismatch)
I20240502 14:00:19.700256 30500 consensus_queue.cc:1035] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 211, Last known committed idx: 210, Time since last communication: 0.000s
I20240502 14:00:19.700867 30500 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P 0280081e4ff74896832ec3985c372b3a [term 1 LEADER]: Committing config change with OpId 1.211: config changed from index 210 to 211, VOTER b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68) evicted. New config: { opid_index: 211 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.701278 27519 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.211: config changed from index 210 to 211, VOTER b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68) evicted. New config: { opid_index: 211 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.701455 26985 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.211: config changed from index 210 to 211, VOTER b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68) evicted. New config: { opid_index: 211 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.702476 26703 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4a74c0c312a7403d89958a3632cab7d3 with cas_config_opid_index 210: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:19.702963 27381 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P e44f8c16a14a48a38a5a74af2899c217 [term 1 FOLLOWER]: Committing config change with OpId 1.211: config changed from index 210 to 211, VOTER b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68) evicted. New config: { opid_index: 211 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.703835 29609 raft_consensus.cc:2932] T 4a74c0c312a7403d89958a3632cab7d3 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Committing config change with OpId 1.211: config changed from index 210 to 211, VOTER b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68) evicted. New config: { opid_index: 211 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.704349 26708 catalog_manager.cc:5484] T 4a74c0c312a7403d89958a3632cab7d3 P 256ce64303ec49e1a5cd8e0ffee436b0 reported cstate change: config changed from index 210 to 211, VOTER b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "0280081e4ff74896832ec3985c372b3a" committed_config { opid_index: 211 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "0280081e4ff74896832ec3985c372b3a" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 35733 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { promote: false } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.706719 27234 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a74c0c312a7403d89958a3632cab7d3 with delete_type TABLET_DATA_TOMBSTONED (TS b4fc3c795b1d4505b3a2903532d38b52 not found in new config with opid_index 211) from {username='slave'} at 127.0.0.1:57834
I20240502 14:00:19.707582 30509 tablet_replica.cc:330] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52: stopping tablet replica
I20240502 14:00:19.707875 30509 raft_consensus.cc:2229] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:19.708083 30509 raft_consensus.cc:2258] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:19.709738 30509 ts_tablet_manager.cc:1875] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:19.711356 30509 ts_tablet_manager.cc:1888] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.210
I20240502 14:00:19.711418 30509 log.cc:1196] T 4a74c0c312a7403d89958a3632cab7d3 P b4fc3c795b1d4505b3a2903532d38b52: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/4a74c0c312a7403d89958a3632cab7d3
I20240502 14:00:19.711865 26703 catalog_manager.cc:4830] TS b4fc3c795b1d4505b3a2903532d38b52 (127.24.66.68:35167): tablet 4a74c0c312a7403d89958a3632cab7d3 (table rebalance_test_table_1 [id=a35cd53b479c486fad6a93e2f4317542]) successfully deleted
I20240502 14:00:19.721451 30482 raft_consensus.cc:1050] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217: attempting to promote NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 to VOTER
I20240502 14:00:19.721854 30482 consensus_queue.cc:237] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 268, Committed index: 268, Last appended: 1.268, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 269 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } }
I20240502 14:00:19.723270 29609 raft_consensus.cc:1261] T d6fcbe92870d40a6b4a860f5f4f81c29 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 268. Preceding OpId from leader: term: 1 index: 269. (index mismatch)
I20240502 14:00:19.723316 26985 raft_consensus.cc:1261] T d6fcbe92870d40a6b4a860f5f4f81c29 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 268. Preceding OpId from leader: term: 1 index: 269. (index mismatch)
I20240502 14:00:19.723385 26858 raft_consensus.cc:1261] T d6fcbe92870d40a6b4a860f5f4f81c29 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 268. Preceding OpId from leader: term: 1 index: 269. (index mismatch)
I20240502 14:00:19.723583 27254 raft_consensus.cc:1261] T d6fcbe92870d40a6b4a860f5f4f81c29 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 268. Preceding OpId from leader: term: 1 index: 269. (index mismatch)
I20240502 14:00:19.723863 30513 consensus_queue.cc:1035] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 269, Last known committed idx: 268, Time since last communication: 0.000s
I20240502 14:00:19.724040 30438 consensus_queue.cc:1035] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 269, Last known committed idx: 268, Time since last communication: 0.000s
I20240502 14:00:19.724241 30482 consensus_queue.cc:1035] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 269, Last known committed idx: 268, Time since last communication: 0.000s
I20240502 14:00:19.724404 27519 raft_consensus.cc:1261] T d6fcbe92870d40a6b4a860f5f4f81c29 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 268. Preceding OpId from leader: term: 1 index: 269. (index mismatch)
I20240502 14:00:19.724534 29953 consensus_queue.cc:1035] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 269, Last known committed idx: 268, Time since last communication: 0.000s
I20240502 14:00:19.724824 30482 consensus_queue.cc:1035] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 269, Last known committed idx: 268, Time since last communication: 0.000s
I20240502 14:00:19.726585 30438 raft_consensus.cc:2932] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [term 1 LEADER]: Committing config change with OpId 1.269: config changed from index 268 to 269, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 269 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.727027 27519 raft_consensus.cc:2932] T d6fcbe92870d40a6b4a860f5f4f81c29 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.269: config changed from index 268 to 269, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 269 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.727218 27254 raft_consensus.cc:2932] T d6fcbe92870d40a6b4a860f5f4f81c29 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.269: config changed from index 268 to 269, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 269 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.727943 26858 raft_consensus.cc:2932] T d6fcbe92870d40a6b4a860f5f4f81c29 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.269: config changed from index 268 to 269, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 269 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.729780 26708 catalog_manager.cc:5484] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 reported cstate change: config changed from index 268 to 269, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e44f8c16a14a48a38a5a74af2899c217" committed_config { opid_index: 269 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:19.729804 29609 raft_consensus.cc:2932] T d6fcbe92870d40a6b4a860f5f4f81c29 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Committing config change with OpId 1.269: config changed from index 268 to 269, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 269 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.731321 26985 raft_consensus.cc:2932] T d6fcbe92870d40a6b4a860f5f4f81c29 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.269: config changed from index 268 to 269, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 269 OBSOLETE_local: false peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } attrs { replace: true } } peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.732956 27381 consensus_queue.cc:237] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 269, Committed index: 269, Last appended: 1.269, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 270 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } }
I20240502 14:00:19.734393 26858 raft_consensus.cc:1261] T d6fcbe92870d40a6b4a860f5f4f81c29 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 269. Preceding OpId from leader: term: 1 index: 270. (index mismatch)
I20240502 14:00:19.734786 29609 raft_consensus.cc:1261] T d6fcbe92870d40a6b4a860f5f4f81c29 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 269. Preceding OpId from leader: term: 1 index: 270. (index mismatch)
I20240502 14:00:19.735002 27254 raft_consensus.cc:1261] T d6fcbe92870d40a6b4a860f5f4f81c29 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 269. Preceding OpId from leader: term: 1 index: 270. (index mismatch)
I20240502 14:00:19.735113 26985 raft_consensus.cc:1261] T d6fcbe92870d40a6b4a860f5f4f81c29 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 269. Preceding OpId from leader: term: 1 index: 270. (index mismatch)
I20240502 14:00:19.735134 30438 consensus_queue.cc:1035] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 270, Last known committed idx: 269, Time since last communication: 0.000s
I20240502 14:00:19.735471 30438 consensus_queue.cc:1035] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 270, Last known committed idx: 269, Time since last communication: 0.000s
I20240502 14:00:19.735723 30513 consensus_queue.cc:1035] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 270, Last known committed idx: 269, Time since last communication: 0.000s
I20240502 14:00:19.736157 30513 consensus_queue.cc:1035] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 270, Last known committed idx: 269, Time since last communication: 0.000s
I20240502 14:00:19.737357 30438 raft_consensus.cc:2932] T d6fcbe92870d40a6b4a860f5f4f81c29 P e44f8c16a14a48a38a5a74af2899c217 [term 1 LEADER]: Committing config change with OpId 1.270: config changed from index 269 to 270, VOTER 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70) evicted. New config: { opid_index: 270 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.737748 26985 raft_consensus.cc:2932] T d6fcbe92870d40a6b4a860f5f4f81c29 P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.270: config changed from index 269 to 270, VOTER 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70) evicted. New config: { opid_index: 270 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.737980 29609 raft_consensus.cc:2932] T d6fcbe92870d40a6b4a860f5f4f81c29 P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Committing config change with OpId 1.270: config changed from index 269 to 270, VOTER 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70) evicted. New config: { opid_index: 270 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.739094 26701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d6fcbe92870d40a6b4a860f5f4f81c29 with cas_config_opid_index 269: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:19.739919 26858 raft_consensus.cc:2932] T d6fcbe92870d40a6b4a860f5f4f81c29 P d3e36186faa442b7bd87357e74d47f68 [term 1 FOLLOWER]: Committing config change with OpId 1.270: config changed from index 269 to 270, VOTER 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70) evicted. New config: { opid_index: 270 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.740291 26712 catalog_manager.cc:5484] T d6fcbe92870d40a6b4a860f5f4f81c29 P 256ce64303ec49e1a5cd8e0ffee436b0 reported cstate change: config changed from index 269 to 270, VOTER 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70) evicted. New cstate: current_term: 1 leader_uuid: "e44f8c16a14a48a38a5a74af2899c217" committed_config { opid_index: 270 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.740532 27254 raft_consensus.cc:2932] T d6fcbe92870d40a6b4a860f5f4f81c29 P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.270: config changed from index 269 to 270, VOTER 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70) evicted. New config: { opid_index: 270 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "d3e36186faa442b7bd87357e74d47f68" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40841 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:19.743182 27499 tablet_service.cc:1514] Processing DeleteTablet for tablet d6fcbe92870d40a6b4a860f5f4f81c29 with delete_type TABLET_DATA_TOMBSTONED (TS 34b161bd719d4d72baf8da4f40aef136 not found in new config with opid_index 270) from {username='slave'} at 127.0.0.1:33294
I20240502 14:00:19.743537 30519 tablet_replica.cc:330] T d6fcbe92870d40a6b4a860f5f4f81c29 P 34b161bd719d4d72baf8da4f40aef136: stopping tablet replica
I20240502 14:00:19.743727 30519 raft_consensus.cc:2229] T d6fcbe92870d40a6b4a860f5f4f81c29 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:19.743945 30519 raft_consensus.cc:2258] T d6fcbe92870d40a6b4a860f5f4f81c29 P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:19.745976 30519 ts_tablet_manager.cc:1875] T d6fcbe92870d40a6b4a860f5f4f81c29 P 34b161bd719d4d72baf8da4f40aef136: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:19.747632 30519 ts_tablet_manager.cc:1888] T d6fcbe92870d40a6b4a860f5f4f81c29 P 34b161bd719d4d72baf8da4f40aef136: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.269
I20240502 14:00:19.747694 30519 log.cc:1196] T d6fcbe92870d40a6b4a860f5f4f81c29 P 34b161bd719d4d72baf8da4f40aef136: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal/wals/d6fcbe92870d40a6b4a860f5f4f81c29
I20240502 14:00:19.748155 26700 catalog_manager.cc:4830] TS 34b161bd719d4d72baf8da4f40aef136 (127.24.66.70:38217): tablet d6fcbe92870d40a6b4a860f5f4f81c29 (table rebalance_test_table_4 [id=56abfd3d24fd434b996a35f2174a0641]) successfully deleted
I20240502 14:00:20.150609 30482 raft_consensus.cc:1050] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217: attempting to promote NON_VOTER b15496b8de1b4bc084cdfbb7c581f361 to VOTER
I20240502 14:00:20.151021 30482 consensus_queue.cc:237] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } }
I20240502 14:00:20.152496 29609 raft_consensus.cc:1261] T 4a156115301a4d40bfc7f2bd11f135bf P b15496b8de1b4bc084cdfbb7c581f361 [term 1 LEARNER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:20.152576 27254 raft_consensus.cc:1261] T 4a156115301a4d40bfc7f2bd11f135bf P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:20.152745 26985 raft_consensus.cc:1261] T 4a156115301a4d40bfc7f2bd11f135bf P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:20.152981 30513 consensus_queue.cc:1035] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:20.153172 30546 consensus_queue.cc:1035] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:20.153214 28363 raft_consensus.cc:1261] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:20.153398 27519 raft_consensus.cc:1261] T 4a156115301a4d40bfc7f2bd11f135bf P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:00:20.153704 30546 consensus_queue.cc:1035] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:20.153889 30513 consensus_queue.cc:1035] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:20.154713 30546 consensus_queue.cc:1035] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 14:00:20.156298 30511 raft_consensus.cc:2932] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:20.156672 29609 raft_consensus.cc:2932] T 4a156115301a4d40bfc7f2bd11f135bf P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:20.156863 27254 raft_consensus.cc:2932] T 4a156115301a4d40bfc7f2bd11f135bf P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:20.159080 28363 raft_consensus.cc:2932] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:20.159497 26713 catalog_manager.cc:5484] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 reported cstate change: config changed from index 2 to 3, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "e44f8c16a14a48a38a5a74af2899c217" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:20.160377 26985 raft_consensus.cc:2932] T 4a156115301a4d40bfc7f2bd11f135bf P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:20.161304 27519 raft_consensus.cc:2932] T 4a156115301a4d40bfc7f2bd11f135bf P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, b15496b8de1b4bc084cdfbb7c581f361 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "256ce64303ec49e1a5cd8e0ffee436b0" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 40473 } attrs { replace: true } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:20.163908 27381 consensus_queue.cc:237] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } }
I20240502 14:00:20.165347 28363 raft_consensus.cc:1261] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:20.165591 27254 raft_consensus.cc:1261] T 4a156115301a4d40bfc7f2bd11f135bf P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:20.165753 27519 raft_consensus.cc:1261] T 4a156115301a4d40bfc7f2bd11f135bf P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:20.165694 30546 consensus_queue.cc:1035] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:20.165918 30482 consensus_queue.cc:1035] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:20.166049 29609 raft_consensus.cc:1261] T 4a156115301a4d40bfc7f2bd11f135bf P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Refusing update from remote peer e44f8c16a14a48a38a5a74af2899c217: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:20.166260 29953 consensus_queue.cc:1035] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:20.166675 29953 consensus_queue.cc:1035] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:20.167845 29953 raft_consensus.cc:2932] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:20.168370 27254 raft_consensus.cc:2932] T 4a156115301a4d40bfc7f2bd11f135bf P b4fc3c795b1d4505b3a2903532d38b52 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:20.169209 29609 raft_consensus.cc:2932] T 4a156115301a4d40bfc7f2bd11f135bf P b15496b8de1b4bc084cdfbb7c581f361 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:20.170156 26708 catalog_manager.cc:5484] T 4a156115301a4d40bfc7f2bd11f135bf P e44f8c16a14a48a38a5a74af2899c217 reported cstate change: config changed from index 3 to 4, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New cstate: current_term: 1 leader_uuid: "e44f8c16a14a48a38a5a74af2899c217" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:20.170398 28363 raft_consensus.cc:2932] T 4a156115301a4d40bfc7f2bd11f135bf P 5f9ddfa1f6fc484d8fb96075a8efb649 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:20.172370 27519 raft_consensus.cc:2932] T 4a156115301a4d40bfc7f2bd11f135bf P 34b161bd719d4d72baf8da4f40aef136 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b4fc3c795b1d4505b3a2903532d38b52" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 35167 } } peers { permanent_uuid: "34b161bd719d4d72baf8da4f40aef136" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38217 } } peers { permanent_uuid: "5f9ddfa1f6fc484d8fb96075a8efb649" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 42201 } } peers { permanent_uuid: "e44f8c16a14a48a38a5a74af2899c217" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 39107 } } peers { permanent_uuid: "b15496b8de1b4bc084cdfbb7c581f361" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 46715 } attrs { promote: false } } }
I20240502 14:00:20.172655 26971 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a156115301a4d40bfc7f2bd11f135bf with delete_type TABLET_DATA_TOMBSTONED (TS 256ce64303ec49e1a5cd8e0ffee436b0 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51832
I20240502 14:00:20.174008 26701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4a156115301a4d40bfc7f2bd11f135bf with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:20.174561 30553 tablet_replica.cc:330] T 4a156115301a4d40bfc7f2bd11f135bf P 256ce64303ec49e1a5cd8e0ffee436b0: stopping tablet replica
I20240502 14:00:20.174706 30553 raft_consensus.cc:2229] T 4a156115301a4d40bfc7f2bd11f135bf P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:20.174858 30553 raft_consensus.cc:2258] T 4a156115301a4d40bfc7f2bd11f135bf P 256ce64303ec49e1a5cd8e0ffee436b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:20.176417 30553 ts_tablet_manager.cc:1875] T 4a156115301a4d40bfc7f2bd11f135bf P 256ce64303ec49e1a5cd8e0ffee436b0: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:20.178100 30553 ts_tablet_manager.cc:1888] T 4a156115301a4d40bfc7f2bd11f135bf P 256ce64303ec49e1a5cd8e0ffee436b0: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20240502 14:00:20.178164 30553 log.cc:1196] T 4a156115301a4d40bfc7f2bd11f135bf P 256ce64303ec49e1a5cd8e0ffee436b0: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/wals/4a156115301a4d40bfc7f2bd11f135bf
I20240502 14:00:20.178612 26700 catalog_manager.cc:4830] TS 256ce64303ec49e1a5cd8e0ffee436b0 (127.24.66.66:40473): tablet 4a156115301a4d40bfc7f2bd11f135bf (table TestTable [id=beb77348bd534888844c42de955fc55b]) successfully deleted
I20240502 14:00:20.359509 27101 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:20.359988 29588 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:20.365479 27367 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:00:20.366040 27234 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:00:20.382675 26837 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:20.392717 28345 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:20.393007 27499 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:00:20.395339 26971 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:20.478881 27101 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:20.480289 29588 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:20.482029 27234 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:00:20.484387 26971 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:20.487185 28345 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:20.506680 27499 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:00:20.507234 27367 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:00:20.522377 26837 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:20.613401 27101 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 14:00:20.619139 27367 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:00:20.620476 28345 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:20.620479 27499 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:00:20.630857 27234 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:00:20.640081 26837 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:20.642064 26971 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:20.644982 29588 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
60bad1ef960c4e09b6fb0fe11be17da2 | 127.24.66.126:45215 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
allow_unsafe_replication_factor | true | runtime,unsafe | all 1 server(s) checked
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.24.66.84:37497 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
0280081e4ff74896832ec3985c372b3a | 127.24.66.67:35733 | HEALTHY | <none> | 2 | 0
256ce64303ec49e1a5cd8e0ffee436b0 | 127.24.66.66:40473 | HEALTHY | <none> | 1 | 0
34b161bd719d4d72baf8da4f40aef136 | 127.24.66.70:38217 | HEALTHY | <none> | 3 | 0
5f9ddfa1f6fc484d8fb96075a8efb649 | 127.24.66.71:42201 | HEALTHY | <none> | 0 | 0
b15496b8de1b4bc084cdfbb7c581f361 | 127.24.66.72:46715 | HEALTHY | <none> | 0 | 0
b4fc3c795b1d4505b3a2903532d38b52 | 127.24.66.68:35167 | HEALTHY | <none> | 5 | 0
d3e36186faa442b7bd87357e74d47f68 | 127.24.66.65:40841 | HEALTHY | <none> | 1 | 0
e44f8c16a14a48a38a5a74af2899c217 | 127.24.66.69:39107 | HEALTHY | <none> | 4 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 8
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 8 server(s) checked
local_ip_for_outbound_sockets | 127.24.66.65 | experimental | 127.24.66.65:40841
local_ip_for_outbound_sockets | 127.24.66.66 | experimental | 127.24.66.66:40473
local_ip_for_outbound_sockets | 127.24.66.67 | experimental | 127.24.66.67:35733
local_ip_for_outbound_sockets | 127.24.66.68 | experimental | 127.24.66.68:35167
local_ip_for_outbound_sockets | 127.24.66.69 | experimental | 127.24.66.69:39107
local_ip_for_outbound_sockets | 127.24.66.70 | experimental | 127.24.66.70:38217
local_ip_for_outbound_sockets | 127.24.66.71 | experimental | 127.24.66.71:42201
local_ip_for_outbound_sockets | 127.24.66.72 | experimental | 127.24.66.72:46715
never_fsync | true | unsafe,advanced | all 8 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 8 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 8 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 8 server(s) checked
server_dump_info_format | pb | hidden | all 8 server(s) checked
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.24.66.65:40841
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.24.66.66:40473
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.24.66.67:35733
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.24.66.68:35167
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.24.66.69:39107
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.24.66.70:38217
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.24.66.71:42201
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalanceParamTest.Rebalance_7.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data/info.pb | hidden | 127.24.66.72:46715
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.24.66.84:37497 | all 8 server(s) checked
time_source | builtin | all 8 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 9 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 5 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 5 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 10
First Quartile | 10
Median | 10
Third Quartile | 10
Maximum | 10
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 8
Tables | 6
Tablets | 16
Replicas | 80
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240502 14:00:20.685571 24841 log_verifier.cc:126] Checking tablet 1ad8b348f02043dfa4736276dc306895
I20240502 14:00:20.839212 24841 log_verifier.cc:177] Verified matching terms for 353 ops in tablet 1ad8b348f02043dfa4736276dc306895
I20240502 14:00:20.839502 24841 log_verifier.cc:126] Checking tablet 41e73be38c4348a5bb86da6c0f6c6d22
I20240502 14:00:20.957618 24841 log_verifier.cc:177] Verified matching terms for 225 ops in tablet 41e73be38c4348a5bb86da6c0f6c6d22
I20240502 14:00:20.957852 24841 log_verifier.cc:126] Checking tablet 4a156115301a4d40bfc7f2bd11f135bf
I20240502 14:00:20.984020 24841 log_verifier.cc:177] Verified matching terms for 4 ops in tablet 4a156115301a4d40bfc7f2bd11f135bf
I20240502 14:00:20.984097 24841 log_verifier.cc:126] Checking tablet 4a74c0c312a7403d89958a3632cab7d3
I20240502 14:00:21.087635 24841 log_verifier.cc:177] Verified matching terms for 211 ops in tablet 4a74c0c312a7403d89958a3632cab7d3
I20240502 14:00:21.087836 24841 log_verifier.cc:126] Checking tablet 53a68b75cbce41d69432f21f2cfbf256
I20240502 14:00:21.225756 24841 log_verifier.cc:177] Verified matching terms for 295 ops in tablet 53a68b75cbce41d69432f21f2cfbf256
I20240502 14:00:21.225975 24841 log_verifier.cc:126] Checking tablet 6ed88ddf0d1b4dabbbf4604019705436
I20240502 14:00:21.369951 24841 log_verifier.cc:177] Verified matching terms for 352 ops in tablet 6ed88ddf0d1b4dabbbf4604019705436
I20240502 14:00:21.370194 24841 log_verifier.cc:126] Checking tablet 945d054a56ed4043b7b49d14ed917532
I20240502 14:00:21.452050 24841 log_verifier.cc:177] Verified matching terms for 154 ops in tablet 945d054a56ed4043b7b49d14ed917532
I20240502 14:00:21.452181 24841 log_verifier.cc:126] Checking tablet 95f064e56ee241dba95f9a6cd5984d97
I20240502 14:00:21.546893 24841 log_verifier.cc:177] Verified matching terms for 196 ops in tablet 95f064e56ee241dba95f9a6cd5984d97
I20240502 14:00:21.547061 24841 log_verifier.cc:126] Checking tablet 9efaa3c68c534606bd37bbe4703b2ab2
I20240502 14:00:21.677691 24841 log_verifier.cc:177] Verified matching terms for 291 ops in tablet 9efaa3c68c534606bd37bbe4703b2ab2
I20240502 14:00:21.677898 24841 log_verifier.cc:126] Checking tablet a734b1d6d928408cac2a9e1b263f2697
I20240502 14:00:21.772166 24841 log_verifier.cc:177] Verified matching terms for 188 ops in tablet a734b1d6d928408cac2a9e1b263f2697
I20240502 14:00:21.772377 24841 log_verifier.cc:126] Checking tablet abb00d05d9704f5e8a78cb9fd1866e2a
I20240502 14:00:21.910037 24841 log_verifier.cc:177] Verified matching terms for 307 ops in tablet abb00d05d9704f5e8a78cb9fd1866e2a
I20240502 14:00:21.910236 24841 log_verifier.cc:126] Checking tablet d6fcbe92870d40a6b4a860f5f4f81c29
I20240502 14:00:22.042878 24841 log_verifier.cc:177] Verified matching terms for 270 ops in tablet d6fcbe92870d40a6b4a860f5f4f81c29
I20240502 14:00:22.043056 24841 log_verifier.cc:126] Checking tablet df55bedff83e4a5d822547e0fa1c2f87
I20240502 14:00:22.128602 24841 log_verifier.cc:177] Verified matching terms for 178 ops in tablet df55bedff83e4a5d822547e0fa1c2f87
I20240502 14:00:22.128734 24841 log_verifier.cc:126] Checking tablet f4bb31e5c0454e54a7b3855445120783
I20240502 14:00:22.269397 24841 log_verifier.cc:177] Verified matching terms for 321 ops in tablet f4bb31e5c0454e54a7b3855445120783
I20240502 14:00:22.269615 24841 log_verifier.cc:126] Checking tablet f7ea57a2ffe04e9e962c61d0e82fc475
I20240502 14:00:22.390781 24841 log_verifier.cc:177] Verified matching terms for 265 ops in tablet f7ea57a2ffe04e9e962c61d0e82fc475
I20240502 14:00:22.390985 24841 log_verifier.cc:126] Checking tablet fcf8a440cfbe490e9aa735cad599e472
I20240502 14:00:22.497956 24841 log_verifier.cc:177] Verified matching terms for 243 ops in tablet fcf8a440cfbe490e9aa735cad599e472
I20240502 14:00:22.499430 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 26777
I20240502 14:00:22.507875 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 26909
I20240502 14:00:22.514541 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 27041
I20240502 14:00:22.521618 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 27173
I20240502 14:00:22.528983 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 27304
I20240502 14:00:22.535364 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 27437
I20240502 14:00:22.542294 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 28279
I20240502 14:00:22.547966 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 29526
I20240502 14:00:22.553376 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 26685
2024-05-02T14:00:22Z chronyd exiting
[ OK ] RebalanceParamTest.Rebalance/7 (34165 ms)
[----------] 1 test from RebalanceParamTest (34165 ms total)
[----------] 1 test from TserverAddedDuringRebalancingTest
[ RUN ] TserverAddedDuringRebalancingTest.TserverStarts/1
I20240502 14:00:22.652046 24841 test_util.cc:268] Using random seed: 103906440
I20240502 14:00:22.652834 24841 ts_itest-base.cc:112] Starting cluster with:
I20240502 14:00:22.652869 24841 ts_itest-base.cc:113] --------------
I20240502 14:00:22.652889 24841 ts_itest-base.cc:114] 8 tablet servers
I20240502 14:00:22.652905 24841 ts_itest-base.cc:115] 3 replicas per TS
I20240502 14:00:22.652928 24841 ts_itest-base.cc:116] --------------
2024-05-02T14:00:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T14:00:22Z Disabled control of system clock
I20240502 14:00:22.660938 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:40727
--webserver_interface=127.24.66.126
--webserver_port=0
--builtin_ntp_servers=127.24.66.84:35285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:40727
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false with env {}
W20240502 14:00:22.758543 30682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:22.758735 30682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:22.758775 30682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:22.761567 30682 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:22.761617 30682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 14:00:22.761637 30682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:22.761664 30682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 14:00:22.761684 30682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 14:00:22.765058 30682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:35285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:40727
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:40727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.30682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:22.765717 30682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:22.766641 30682 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:22.773682 30688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:22.774304 30691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:22.774900 30687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:22.775914 30682 server_base.cc:1034] running on GCE node
I20240502 14:00:22.776302 30682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:22.777065 30682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:22.778203 30682 hybrid_clock.cc:648] HybridClock initialized: now 1714658422778193 us; error 23 us; skew 500 ppm
I20240502 14:00:22.784233 30682 webserver.cc:460] Webserver started at http://127.24.66.126:40153/ using document root <none> and password file <none>
I20240502 14:00:22.784824 30682 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:22.784886 30682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:22.785075 30682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:22.786481 30682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "2085e95de92d409c8396066e617c592b"
format_stamp: "Formatted at 2024-05-02 14:00:22 on dist-test-slave-5npg"
I20240502 14:00:22.786904 30682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "2085e95de92d409c8396066e617c592b"
format_stamp: "Formatted at 2024-05-02 14:00:22 on dist-test-slave-5npg"
I20240502 14:00:22.789901 30682 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 14:00:22.791738 30697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:22.792593 30682 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:22.792699 30682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
uuid: "2085e95de92d409c8396066e617c592b"
format_stamp: "Formatted at 2024-05-02 14:00:22 on dist-test-slave-5npg"
I20240502 14:00:22.792773 30682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:22.809839 30682 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:22.810312 30682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:22.810446 30682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:22.816433 30753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:40727 every 8 connection(s)
I20240502 14:00:22.816417 30682 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:40727
I20240502 14:00:22.817253 30682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 14:00:22.819473 30754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:22.823871 30754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2085e95de92d409c8396066e617c592b: Bootstrap starting.
I20240502 14:00:22.825387 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 30682
I20240502 14:00:22.825515 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 14:00:22.825845 30754 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2085e95de92d409c8396066e617c592b: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:22.826552 30754 log.cc:824] T 00000000000000000000000000000000 P 2085e95de92d409c8396066e617c592b: Log is configured to *not* fsync() on all Append() calls
I20240502 14:00:22.828006 30754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2085e95de92d409c8396066e617c592b: No bootstrap required, opened a new log
I20240502 14:00:22.830806 30754 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2085e95de92d409c8396066e617c592b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2085e95de92d409c8396066e617c592b" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40727 } }
I20240502 14:00:22.831000 30754 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2085e95de92d409c8396066e617c592b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:22.831075 30754 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2085e95de92d409c8396066e617c592b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2085e95de92d409c8396066e617c592b, State: Initialized, Role: FOLLOWER
I20240502 14:00:22.831468 30754 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2085e95de92d409c8396066e617c592b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2085e95de92d409c8396066e617c592b" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40727 } }
I20240502 14:00:22.831604 30754 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2085e95de92d409c8396066e617c592b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 14:00:22.831713 30754 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2085e95de92d409c8396066e617c592b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 14:00:22.831866 30754 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2085e95de92d409c8396066e617c592b [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:22.832553 30754 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2085e95de92d409c8396066e617c592b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2085e95de92d409c8396066e617c592b" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40727 } }
I20240502 14:00:22.832842 30754 leader_election.cc:302] T 00000000000000000000000000000000 P 2085e95de92d409c8396066e617c592b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2085e95de92d409c8396066e617c592b; no voters:
I20240502 14:00:22.833071 30754 leader_election.cc:288] T 00000000000000000000000000000000 P 2085e95de92d409c8396066e617c592b [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 14:00:22.833165 30759 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2085e95de92d409c8396066e617c592b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:22.833787 30759 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2085e95de92d409c8396066e617c592b [term 1 LEADER]: Becoming Leader. State: Replica: 2085e95de92d409c8396066e617c592b, State: Running, Role: LEADER
I20240502 14:00:22.834055 30754 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2085e95de92d409c8396066e617c592b [sys.catalog]: configured and running, proceeding with master startup.
I20240502 14:00:22.834061 30759 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2085e95de92d409c8396066e617c592b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2085e95de92d409c8396066e617c592b" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40727 } }
I20240502 14:00:22.835031 30761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2085e95de92d409c8396066e617c592b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2085e95de92d409c8396066e617c592b. Latest consensus state: current_term: 1 leader_uuid: "2085e95de92d409c8396066e617c592b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2085e95de92d409c8396066e617c592b" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40727 } } }
I20240502 14:00:22.835144 30761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2085e95de92d409c8396066e617c592b [sys.catalog]: This master's current role is: LEADER
I20240502 14:00:22.835352 30760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2085e95de92d409c8396066e617c592b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2085e95de92d409c8396066e617c592b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2085e95de92d409c8396066e617c592b" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40727 } } }
I20240502 14:00:22.835440 30760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2085e95de92d409c8396066e617c592b [sys.catalog]: This master's current role is: LEADER
I20240502 14:00:22.835690 30768 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 14:00:22.838438 30768 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 14:00:22.842881 30768 catalog_manager.cc:1347] Generated new cluster ID: 877af9580b7a4ff2a409de5cecba19cb
I20240502 14:00:22.842947 30768 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 14:00:22.880059 30768 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 14:00:22.881050 30768 catalog_manager.cc:1504] Loading token signing keys...
I20240502 14:00:22.897817 30768 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 2085e95de92d409c8396066e617c592b: Generated new TSK 0
I20240502 14:00:22.898438 30768 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 14:00:22.910336 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40727
--builtin_ntp_servers=127.24.66.84:35285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
W20240502 14:00:23.026615 30779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:23.026805 30779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:23.026873 30779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:23.029752 30779 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:23.029820 30779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:23.029911 30779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20240502 14:00:23.033316 30779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:35285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.24.66.126:40727
--never_fsync=true
--heap_profile_path=/tmp/kudu.30779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:23.034097 30779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:23.035176 30779 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:23.043334 30785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:23.043928 30787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:23.044559 30784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:23.045521 30779 server_base.cc:1034] running on GCE node
I20240502 14:00:23.045833 30779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:23.046249 30779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:23.047385 30779 hybrid_clock.cc:648] HybridClock initialized: now 1714658423047362 us; error 35 us; skew 500 ppm
I20240502 14:00:23.049158 30779 webserver.cc:460] Webserver started at http://127.24.66.65:45927/ using document root <none> and password file <none>
I20240502 14:00:23.049635 30779 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:23.049690 30779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:23.049841 30779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:23.051354 30779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
format_stamp: "Formatted at 2024-05-02 14:00:23 on dist-test-slave-5npg"
I20240502 14:00:23.051827 30779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
format_stamp: "Formatted at 2024-05-02 14:00:23 on dist-test-slave-5npg"
I20240502 14:00:23.054814 30779 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 14:00:23.056737 30793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:23.057649 30779 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:23.057761 30779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
format_stamp: "Formatted at 2024-05-02 14:00:23 on dist-test-slave-5npg"
I20240502 14:00:23.057833 30779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:23.078898 30779 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:23.079406 30779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:23.079564 30779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:23.080045 30779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:23.080857 30779 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:23.080904 30779 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:23.080957 30779 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:23.080986 30779 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:23.089073 30779 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:39553
I20240502 14:00:23.089100 30913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:39553 every 8 connection(s)
I20240502 14:00:23.089947 30779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 14:00:23.096379 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 30779
I20240502 14:00:23.096504 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 14:00:23.098066 30914 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40727
I20240502 14:00:23.098253 30914 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:23.098606 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40727
--builtin_ntp_servers=127.24.66.84:35285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240502 14:00:23.098731 30914 heartbeater.cc:511] Master 127.24.66.126:40727 requested a full tablet report, sending...
I20240502 14:00:23.100095 30715 ts_manager.cc:194] Registered new tserver with Master: ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
I20240502 14:00:23.101296 30715 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.65:44595
W20240502 14:00:23.199791 30917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:23.199978 30917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:23.200023 30917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:23.202881 30917 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:23.202935 30917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:23.202999 30917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20240502 14:00:23.206429 30917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:35285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.24.66.126:40727
--never_fsync=true
--heap_profile_path=/tmp/kudu.30917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:23.207157 30917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:23.208151 30917 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:23.216383 30922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:23.217309 30926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:23.217936 30924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:23.218668 30917 server_base.cc:1034] running on GCE node
I20240502 14:00:23.219003 30917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:23.219489 30917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:23.220644 30917 hybrid_clock.cc:648] HybridClock initialized: now 1714658423220632 us; error 26 us; skew 500 ppm
I20240502 14:00:23.222357 30917 webserver.cc:460] Webserver started at http://127.24.66.66:34763/ using document root <none> and password file <none>
I20240502 14:00:23.222872 30917 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:23.222930 30917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:23.223089 30917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:23.224704 30917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "1fab10dcfb8542fca2b965fed6c67929"
format_stamp: "Formatted at 2024-05-02 14:00:23 on dist-test-slave-5npg"
I20240502 14:00:23.225142 30917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "1fab10dcfb8542fca2b965fed6c67929"
format_stamp: "Formatted at 2024-05-02 14:00:23 on dist-test-slave-5npg"
I20240502 14:00:23.228248 30917 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 14:00:23.230206 30932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:23.231068 30917 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:23.231166 30917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "1fab10dcfb8542fca2b965fed6c67929"
format_stamp: "Formatted at 2024-05-02 14:00:23 on dist-test-slave-5npg"
I20240502 14:00:23.231242 30917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:23.267432 30917 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:23.268018 30917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:23.268174 30917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:23.268728 30917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:23.269542 30917 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:23.269599 30917 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:23.269655 30917 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:23.269688 30917 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:23.277809 30917 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:36395
I20240502 14:00:23.277843 31048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:36395 every 8 connection(s)
I20240502 14:00:23.278657 30917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 14:00:23.285637 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 30917
I20240502 14:00:23.285748 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 14:00:23.286316 31049 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40727
I20240502 14:00:23.286504 31049 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:23.286979 31049 heartbeater.cc:511] Master 127.24.66.126:40727 requested a full tablet report, sending...
I20240502 14:00:23.287717 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40727
--builtin_ntp_servers=127.24.66.84:35285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240502 14:00:23.287889 30715 ts_manager.cc:194] Registered new tserver with Master: 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:23.288436 30715 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.66:43177
W20240502 14:00:23.386133 31052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:23.386301 31052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:23.386343 31052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:23.389091 31052 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:23.389139 31052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:23.389204 31052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20240502 14:00:23.392583 31052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:35285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.24.66.126:40727
--never_fsync=true
--heap_profile_path=/tmp/kudu.31052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:23.393256 31052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:23.394188 31052 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:23.402202 31057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:23.403188 31058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:23.403733 31060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:23.404495 31052 server_base.cc:1034] running on GCE node
I20240502 14:00:23.404794 31052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:23.405234 31052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:23.406374 31052 hybrid_clock.cc:648] HybridClock initialized: now 1714658423406352 us; error 34 us; skew 500 ppm
I20240502 14:00:23.407967 31052 webserver.cc:460] Webserver started at http://127.24.66.67:34183/ using document root <none> and password file <none>
I20240502 14:00:23.408447 31052 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:23.408499 31052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:23.408636 31052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:23.410065 31052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "8c61042437444f5f92c76109dfb229fc"
format_stamp: "Formatted at 2024-05-02 14:00:23 on dist-test-slave-5npg"
I20240502 14:00:23.410455 31052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "8c61042437444f5f92c76109dfb229fc"
format_stamp: "Formatted at 2024-05-02 14:00:23 on dist-test-slave-5npg"
I20240502 14:00:23.413380 31052 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 14:00:23.415235 31066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:23.416078 31052 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:23.416162 31052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "8c61042437444f5f92c76109dfb229fc"
format_stamp: "Formatted at 2024-05-02 14:00:23 on dist-test-slave-5npg"
I20240502 14:00:23.416275 31052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:23.437763 31052 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:23.438227 31052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:23.438369 31052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:23.438813 31052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:23.439561 31052 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:23.439605 31052 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:23.439659 31052 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:23.439700 31052 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:23.447191 31052 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:34193
I20240502 14:00:23.447217 31180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:34193 every 8 connection(s)
I20240502 14:00:23.447950 31052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 14:00:23.453133 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 31052
I20240502 14:00:23.453239 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 14:00:23.455163 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.68:0
--local_ip_for_outbound_sockets=127.24.66.68
--webserver_interface=127.24.66.68
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40727
--builtin_ntp_servers=127.24.66.84:35285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240502 14:00:23.455691 31181 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40727
I20240502 14:00:23.455889 31181 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:23.456441 31181 heartbeater.cc:511] Master 127.24.66.126:40727 requested a full tablet report, sending...
I20240502 14:00:23.457407 30715 ts_manager.cc:194] Registered new tserver with Master: 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:23.458117 30715 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.67:40815
W20240502 14:00:23.553772 31184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:23.553962 31184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:23.554008 31184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:23.557052 31184 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:23.557112 31184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:23.557178 31184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.68
I20240502 14:00:23.560703 31184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:35285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.24.66.68
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.24.66.126:40727
--never_fsync=true
--heap_profile_path=/tmp/kudu.31184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.68
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:23.561492 31184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:23.562530 31184 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:23.570720 31189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:23.571288 31190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:23.571925 31192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:23.572893 31184 server_base.cc:1034] running on GCE node
I20240502 14:00:23.573197 31184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:23.573622 31184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:23.574766 31184 hybrid_clock.cc:648] HybridClock initialized: now 1714658423574745 us; error 34 us; skew 500 ppm
I20240502 14:00:23.576434 31184 webserver.cc:460] Webserver started at http://127.24.66.68:43563/ using document root <none> and password file <none>
I20240502 14:00:23.576980 31184 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:23.577032 31184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:23.577188 31184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:23.578744 31184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "14753c93125941acaad7283edc669052"
format_stamp: "Formatted at 2024-05-02 14:00:23 on dist-test-slave-5npg"
I20240502 14:00:23.579164 31184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "14753c93125941acaad7283edc669052"
format_stamp: "Formatted at 2024-05-02 14:00:23 on dist-test-slave-5npg"
I20240502 14:00:23.582162 31184 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20240502 14:00:23.584067 31198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:23.585101 31184 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:23.585192 31184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "14753c93125941acaad7283edc669052"
format_stamp: "Formatted at 2024-05-02 14:00:23 on dist-test-slave-5npg"
I20240502 14:00:23.585294 31184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:23.596449 31184 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:23.597059 31184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:23.597213 31184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:23.597798 31184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:23.598644 31184 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:23.598690 31184 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:23.598773 31184 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:23.598809 31184 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:23.607232 31184 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.68:37029
I20240502 14:00:23.607254 31315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.68:37029 every 8 connection(s)
I20240502 14:00:23.608155 31184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 14:00:23.610880 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 31184
I20240502 14:00:23.611014 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 14:00:23.613025 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.69:0
--local_ip_for_outbound_sockets=127.24.66.69
--webserver_interface=127.24.66.69
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40727
--builtin_ntp_servers=127.24.66.84:35285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240502 14:00:23.616715 31316 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40727
I20240502 14:00:23.616919 31316 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:23.617434 31316 heartbeater.cc:511] Master 127.24.66.126:40727 requested a full tablet report, sending...
I20240502 14:00:23.618477 30715 ts_manager.cc:194] Registered new tserver with Master: 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:23.619073 30715 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.68:34555
W20240502 14:00:23.717231 31319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:23.717468 31319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:23.717542 31319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:23.720494 31319 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:23.720556 31319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:23.720647 31319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.69
I20240502 14:00:23.724021 31319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:35285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.69:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.24.66.69
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.24.66.126:40727
--never_fsync=true
--heap_profile_path=/tmp/kudu.31319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.69
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:23.724793 31319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:23.725816 31319 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:23.733958 31325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:23.734565 31326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:23.735246 31328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:23.736251 31319 server_base.cc:1034] running on GCE node
I20240502 14:00:23.736665 31319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:23.737176 31319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:23.738329 31319 hybrid_clock.cc:648] HybridClock initialized: now 1714658423738317 us; error 26 us; skew 500 ppm
I20240502 14:00:23.740080 31319 webserver.cc:460] Webserver started at http://127.24.66.69:40107/ using document root <none> and password file <none>
I20240502 14:00:23.740645 31319 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:23.740710 31319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:23.740877 31319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:23.742421 31319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "2716cee98b1b4c4591a40e1f466520e5"
format_stamp: "Formatted at 2024-05-02 14:00:23 on dist-test-slave-5npg"
I20240502 14:00:23.742848 31319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "2716cee98b1b4c4591a40e1f466520e5"
format_stamp: "Formatted at 2024-05-02 14:00:23 on dist-test-slave-5npg"
I20240502 14:00:23.745941 31319 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 14:00:23.747853 31334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:23.748826 31319 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:23.748921 31319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "2716cee98b1b4c4591a40e1f466520e5"
format_stamp: "Formatted at 2024-05-02 14:00:23 on dist-test-slave-5npg"
I20240502 14:00:23.748994 31319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:23.780658 31319 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:23.781240 31319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:23.781389 31319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:23.781896 31319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:23.782732 31319 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:23.782779 31319 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:23.782840 31319 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:23.782866 31319 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:23.792694 31319 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.69:35027
I20240502 14:00:23.792706 31447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.69:35027 every 8 connection(s)
I20240502 14:00:23.793705 31319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 14:00:23.799046 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 31319
I20240502 14:00:23.799160 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 14:00:23.801378 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.70:0
--local_ip_for_outbound_sockets=127.24.66.70
--webserver_interface=127.24.66.70
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40727
--builtin_ntp_servers=127.24.66.84:35285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240502 14:00:23.802515 31448 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40727
I20240502 14:00:23.802753 31448 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:23.803387 31448 heartbeater.cc:511] Master 127.24.66.126:40727 requested a full tablet report, sending...
I20240502 14:00:23.804369 30715 ts_manager.cc:194] Registered new tserver with Master: 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69:35027)
I20240502 14:00:23.805107 30715 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.69:34369
W20240502 14:00:23.905018 31451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:23.905205 31451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:23.905247 31451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:23.908087 31451 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:23.908141 31451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:23.908205 31451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.70
I20240502 14:00:23.911480 31451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:35285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.70:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.24.66.70
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.24.66.126:40727
--never_fsync=true
--heap_profile_path=/tmp/kudu.31451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.70
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:23.912153 31451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:23.913125 31451 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:23.921418 31457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:23.921969 31459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:23.922672 31456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:23.923696 31451 server_base.cc:1034] running on GCE node
I20240502 14:00:23.924060 31451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:23.924641 31451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:23.925794 31451 hybrid_clock.cc:648] HybridClock initialized: now 1714658423925767 us; error 41 us; skew 500 ppm
I20240502 14:00:23.927614 31451 webserver.cc:460] Webserver started at http://127.24.66.70:44363/ using document root <none> and password file <none>
I20240502 14:00:23.928156 31451 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:23.928210 31451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:23.928428 31451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:23.930037 31451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "f769c13ece0b40179ba120dbebbaba98"
format_stamp: "Formatted at 2024-05-02 14:00:23 on dist-test-slave-5npg"
I20240502 14:00:23.930478 31451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "f769c13ece0b40179ba120dbebbaba98"
format_stamp: "Formatted at 2024-05-02 14:00:23 on dist-test-slave-5npg"
I20240502 14:00:23.933610 31451 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 14:00:23.935806 31465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:23.936750 31451 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:23.936846 31451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "f769c13ece0b40179ba120dbebbaba98"
format_stamp: "Formatted at 2024-05-02 14:00:23 on dist-test-slave-5npg"
I20240502 14:00:23.936921 31451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:23.953794 31451 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:23.954291 31451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:23.954434 31451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:23.954888 31451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:23.955663 31451 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:23.955716 31451 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:23.955775 31451 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:23.955801 31451 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:23.963996 31584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.70:38291 every 8 connection(s)
I20240502 14:00:23.963979 31451 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.70:38291
I20240502 14:00:23.964969 31451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 14:00:23.967684 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 31451
I20240502 14:00:23.967815 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240502 14:00:23.970145 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.71:0
--local_ip_for_outbound_sockets=127.24.66.71
--webserver_interface=127.24.66.71
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40727
--builtin_ntp_servers=127.24.66.84:35285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240502 14:00:23.973939 31585 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40727
I20240502 14:00:23.974128 31585 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:23.974592 31585 heartbeater.cc:511] Master 127.24.66.126:40727 requested a full tablet report, sending...
I20240502 14:00:23.975503 30715 ts_manager.cc:194] Registered new tserver with Master: f769c13ece0b40179ba120dbebbaba98 (127.24.66.70:38291)
I20240502 14:00:23.976024 30715 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.70:44577
W20240502 14:00:24.075323 31588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:24.075551 31588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:24.075624 31588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:24.078567 31588 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:24.078626 31588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:24.078718 31588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.71
I20240502 14:00:24.082091 31588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:35285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.71:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.24.66.71
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.24.66.126:40727
--never_fsync=true
--heap_profile_path=/tmp/kudu.31588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.71
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:24.083003 31588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:24.084070 31588 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:24.092605 31597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:24.093045 31594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:24.093698 31595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:24.094383 31588 server_base.cc:1034] running on GCE node
I20240502 14:00:24.094714 31588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:24.095255 31588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:24.096386 31588 hybrid_clock.cc:648] HybridClock initialized: now 1714658424096369 us; error 35 us; skew 500 ppm
I20240502 14:00:24.098217 31588 webserver.cc:460] Webserver started at http://127.24.66.71:39219/ using document root <none> and password file <none>
I20240502 14:00:24.098749 31588 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:24.098810 31588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:24.098974 31588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:24.100569 31588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "3daf9e0837c74a919c89e686597ff56b"
format_stamp: "Formatted at 2024-05-02 14:00:24 on dist-test-slave-5npg"
I20240502 14:00:24.101006 31588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "3daf9e0837c74a919c89e686597ff56b"
format_stamp: "Formatted at 2024-05-02 14:00:24 on dist-test-slave-5npg"
I20240502 14:00:24.103646 30914 heartbeater.cc:503] Master 127.24.66.126:40727 was elected leader, sending a full tablet report...
I20240502 14:00:24.104171 31588 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 14:00:24.106357 31603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:24.107372 31588 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:24.107467 31588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "3daf9e0837c74a919c89e686597ff56b"
format_stamp: "Formatted at 2024-05-02 14:00:24 on dist-test-slave-5npg"
I20240502 14:00:24.107559 31588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:24.122874 31588 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:24.123432 31588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:24.123574 31588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:24.124078 31588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:24.124958 31588 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:24.125005 31588 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:24.125047 31588 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:24.125077 31588 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:24.132735 31588 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.71:35161
I20240502 14:00:24.132790 31717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.71:35161 every 8 connection(s)
I20240502 14:00:24.133600 31588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 14:00:24.135809 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 31588
I20240502 14:00:24.135946 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240502 14:00:24.137979 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.72:0
--local_ip_for_outbound_sockets=127.24.66.72
--webserver_interface=127.24.66.72
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40727
--builtin_ntp_servers=127.24.66.84:35285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240502 14:00:24.142045 31718 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40727
I20240502 14:00:24.142280 31718 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:24.142922 31718 heartbeater.cc:511] Master 127.24.66.126:40727 requested a full tablet report, sending...
I20240502 14:00:24.143868 30715 ts_manager.cc:194] Registered new tserver with Master: 3daf9e0837c74a919c89e686597ff56b (127.24.66.71:35161)
I20240502 14:00:24.144389 30715 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.71:60059
W20240502 14:00:24.240592 31721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:24.240785 31721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:24.240851 31721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:24.243928 31721 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:24.243999 31721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:24.244091 31721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.72
I20240502 14:00:24.247567 31721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:35285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.72:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.24.66.72
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.24.66.126:40727
--never_fsync=true
--heap_profile_path=/tmp/kudu.31721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.72
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:24.248500 31721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:24.249574 31721 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:24.257910 31729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:24.258486 31726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:24.259117 31727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:24.259804 31721 server_base.cc:1034] running on GCE node
I20240502 14:00:24.260160 31721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:24.260692 31721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:24.261837 31721 hybrid_clock.cc:648] HybridClock initialized: now 1714658424261817 us; error 35 us; skew 500 ppm
I20240502 14:00:24.263597 31721 webserver.cc:460] Webserver started at http://127.24.66.72:39245/ using document root <none> and password file <none>
I20240502 14:00:24.264170 31721 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:24.264231 31721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:24.264446 31721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:24.266199 31721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "2bb442cde32f426bb14791852dd81802"
format_stamp: "Formatted at 2024-05-02 14:00:24 on dist-test-slave-5npg"
I20240502 14:00:24.266664 31721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "2bb442cde32f426bb14791852dd81802"
format_stamp: "Formatted at 2024-05-02 14:00:24 on dist-test-slave-5npg"
I20240502 14:00:24.269886 31721 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 14:00:24.271915 31735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:24.272897 31721 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:24.272994 31721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "2bb442cde32f426bb14791852dd81802"
format_stamp: "Formatted at 2024-05-02 14:00:24 on dist-test-slave-5npg"
I20240502 14:00:24.273077 31721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:24.286074 31721 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:24.286579 31721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:24.286731 31721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:24.287189 31721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:24.287972 31721 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:24.288022 31721 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:24.288081 31721 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:24.288108 31721 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:24.290186 31049 heartbeater.cc:503] Master 127.24.66.126:40727 was elected leader, sending a full tablet report...
I20240502 14:00:24.296490 31721 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.72:34475
I20240502 14:00:24.296514 31848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.72:34475 every 8 connection(s)
I20240502 14:00:24.297413 31721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 14:00:24.303383 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 31721
I20240502 14:00:24.303503 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240502 14:00:24.305696 31849 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40727
I20240502 14:00:24.305917 31849 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:24.306373 31849 heartbeater.cc:511] Master 127.24.66.126:40727 requested a full tablet report, sending...
I20240502 14:00:24.307226 30715 ts_manager.cc:194] Registered new tserver with Master: 2bb442cde32f426bb14791852dd81802 (127.24.66.72:34475)
I20240502 14:00:24.307729 30715 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.72:35691
I20240502 14:00:24.317612 24841 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240502 14:00:24.326725 24841 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240502 14:00:24.326807 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 31319
I20240502 14:00:24.332059 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 31451
I20240502 14:00:24.337301 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 31588
I20240502 14:00:24.344692 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 31721
I20240502 14:00:24.459857 31181 heartbeater.cc:503] Master 127.24.66.126:40727 was elected leader, sending a full tablet report...
I20240502 14:00:24.621042 31316 heartbeater.cc:503] Master 127.24.66.126:40727 was elected leader, sending a full tablet report...
I20240502 14:00:28.101073 30715 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38902:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:28.120600 30845 tablet_service.cc:1467] Processing CreateTablet for tablet 2e02dc2a98d04bddafdd1b933d89b4dd (DEFAULT_TABLE table=rebalance_test_table_0 [id=a5535db3201345e78ac8467a4d987c08]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.120604 31115 tablet_service.cc:1467] Processing CreateTablet for tablet 2e02dc2a98d04bddafdd1b933d89b4dd (DEFAULT_TABLE table=rebalance_test_table_0 [id=a5535db3201345e78ac8467a4d987c08]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.120649 31245 tablet_service.cc:1467] Processing CreateTablet for tablet 903f5d6c1faf4267b7f7d7dc32670a52 (DEFAULT_TABLE table=rebalance_test_table_0 [id=a5535db3201345e78ac8467a4d987c08]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.121454 31115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e02dc2a98d04bddafdd1b933d89b4dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.121454 31245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 903f5d6c1faf4267b7f7d7dc32670a52. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.121454 30845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e02dc2a98d04bddafdd1b933d89b4dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.121248 30981 tablet_service.cc:1467] Processing CreateTablet for tablet be25ac1c6e7242519ff636242718cb79 (DEFAULT_TABLE table=rebalance_test_table_0 [id=a5535db3201345e78ac8467a4d987c08]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.121927 30981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be25ac1c6e7242519ff636242718cb79. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.124436 31246 tablet_service.cc:1467] Processing CreateTablet for tablet be25ac1c6e7242519ff636242718cb79 (DEFAULT_TABLE table=rebalance_test_table_0 [id=a5535db3201345e78ac8467a4d987c08]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.124593 30842 tablet_service.cc:1467] Processing CreateTablet for tablet 903f5d6c1faf4267b7f7d7dc32670a52 (DEFAULT_TABLE table=rebalance_test_table_0 [id=a5535db3201345e78ac8467a4d987c08]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.124601 30982 tablet_service.cc:1467] Processing CreateTablet for tablet 2e02dc2a98d04bddafdd1b933d89b4dd (DEFAULT_TABLE table=rebalance_test_table_0 [id=a5535db3201345e78ac8467a4d987c08]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.124676 31246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be25ac1c6e7242519ff636242718cb79. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.124715 31114 tablet_service.cc:1467] Processing CreateTablet for tablet 903f5d6c1faf4267b7f7d7dc32670a52 (DEFAULT_TABLE table=rebalance_test_table_0 [id=a5535db3201345e78ac8467a4d987c08]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.124814 30842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 903f5d6c1faf4267b7f7d7dc32670a52. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.124819 30982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e02dc2a98d04bddafdd1b933d89b4dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.124929 31114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 903f5d6c1faf4267b7f7d7dc32670a52. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.128436 30843 tablet_service.cc:1467] Processing CreateTablet for tablet be25ac1c6e7242519ff636242718cb79 (DEFAULT_TABLE table=rebalance_test_table_0 [id=a5535db3201345e78ac8467a4d987c08]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.128674 30843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be25ac1c6e7242519ff636242718cb79. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.130249 31871 tablet_bootstrap.cc:492] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929: Bootstrap starting.
I20240502 14:00:28.130533 31869 tablet_bootstrap.cc:492] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f: Bootstrap starting.
I20240502 14:00:28.131018 31870 tablet_bootstrap.cc:492] T 903f5d6c1faf4267b7f7d7dc32670a52 P 8c61042437444f5f92c76109dfb229fc: Bootstrap starting.
I20240502 14:00:28.132149 31872 tablet_bootstrap.cc:492] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052: Bootstrap starting.
I20240502 14:00:28.132615 31871 tablet_bootstrap.cc:654] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.132793 31870 tablet_bootstrap.cc:654] T 903f5d6c1faf4267b7f7d7dc32670a52 P 8c61042437444f5f92c76109dfb229fc: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.133432 31871 log.cc:824] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929: Log is configured to *not* fsync() on all Append() calls
I20240502 14:00:28.133513 31869 tablet_bootstrap.cc:654] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.133584 31870 log.cc:824] T 903f5d6c1faf4267b7f7d7dc32670a52 P 8c61042437444f5f92c76109dfb229fc: Log is configured to *not* fsync() on all Append() calls
I20240502 14:00:28.133980 31872 tablet_bootstrap.cc:654] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.134696 31872 log.cc:824] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052: Log is configured to *not* fsync() on all Append() calls
I20240502 14:00:28.134773 31869 log.cc:824] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f: Log is configured to *not* fsync() on all Append() calls
I20240502 14:00:28.135332 31870 tablet_bootstrap.cc:492] T 903f5d6c1faf4267b7f7d7dc32670a52 P 8c61042437444f5f92c76109dfb229fc: No bootstrap required, opened a new log
I20240502 14:00:28.135546 31870 ts_tablet_manager.cc:1367] T 903f5d6c1faf4267b7f7d7dc32670a52 P 8c61042437444f5f92c76109dfb229fc: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 14:00:28.136240 31872 tablet_bootstrap.cc:492] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052: No bootstrap required, opened a new log
I20240502 14:00:28.136318 31869 tablet_bootstrap.cc:492] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f: No bootstrap required, opened a new log
I20240502 14:00:28.136483 31869 ts_tablet_manager.cc:1367] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20240502 14:00:28.136500 31872 ts_tablet_manager.cc:1367] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240502 14:00:28.136514 31871 tablet_bootstrap.cc:492] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929: No bootstrap required, opened a new log
I20240502 14:00:28.136636 31871 ts_tablet_manager.cc:1367] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20240502 14:00:28.139215 31870 raft_consensus.cc:348] T 903f5d6c1faf4267b7f7d7dc32670a52 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.139403 31870 raft_consensus.cc:374] T 903f5d6c1faf4267b7f7d7dc32670a52 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.139447 31870 raft_consensus.cc:729] T 903f5d6c1faf4267b7f7d7dc32670a52 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.139496 31869 raft_consensus.cc:348] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.139673 31869 raft_consensus.cc:374] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.139719 31869 raft_consensus.cc:729] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.139878 31870 consensus_queue.cc:260] T 903f5d6c1faf4267b7f7d7dc32670a52 P 8c61042437444f5f92c76109dfb229fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.140165 31871 raft_consensus.cc:348] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.140165 31872 raft_consensus.cc:348] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.140364 31871 raft_consensus.cc:374] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.140364 31872 raft_consensus.cc:374] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.140414 31871 raft_consensus.cc:729] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.140414 31872 raft_consensus.cc:729] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14753c93125941acaad7283edc669052, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.140893 31870 ts_tablet_manager.cc:1398] T 903f5d6c1faf4267b7f7d7dc32670a52 P 8c61042437444f5f92c76109dfb229fc: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240502 14:00:28.140823 31872 consensus_queue.cc:260] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.140823 31871 consensus_queue.cc:260] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.140870 31869 consensus_queue.cc:260] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.141273 31870 tablet_bootstrap.cc:492] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc: Bootstrap starting.
I20240502 14:00:28.141552 31871 ts_tablet_manager.cc:1398] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20240502 14:00:28.141785 31869 ts_tablet_manager.cc:1398] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20240502 14:00:28.141875 31871 tablet_bootstrap.cc:492] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929: Bootstrap starting.
I20240502 14:00:28.142840 31869 tablet_bootstrap.cc:492] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f: Bootstrap starting.
I20240502 14:00:28.142699 31871 tablet_bootstrap.cc:654] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.143235 31870 tablet_bootstrap.cc:654] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.143808 31869 tablet_bootstrap.cc:654] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.144366 31870 tablet_bootstrap.cc:492] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc: No bootstrap required, opened a new log
I20240502 14:00:28.144450 31870 ts_tablet_manager.cc:1367] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:28.144810 31872 ts_tablet_manager.cc:1398] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052: Time spent starting tablet: real 0.008s user 0.002s sys 0.003s
I20240502 14:00:28.144984 31870 raft_consensus.cc:348] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.145072 31871 tablet_bootstrap.cc:492] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929: No bootstrap required, opened a new log
I20240502 14:00:28.145124 31870 raft_consensus.cc:374] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.145144 31871 ts_tablet_manager.cc:1367] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:28.145181 31870 raft_consensus.cc:729] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.145378 31870 consensus_queue.cc:260] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.145735 31870 ts_tablet_manager.cc:1398] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:28.145932 31871 raft_consensus.cc:348] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.145990 31872 tablet_bootstrap.cc:492] T 903f5d6c1faf4267b7f7d7dc32670a52 P 14753c93125941acaad7283edc669052: Bootstrap starting.
I20240502 14:00:28.146054 31871 raft_consensus.cc:374] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.146111 31871 raft_consensus.cc:729] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.146250 31871 consensus_queue.cc:260] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.146649 31871 ts_tablet_manager.cc:1398] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:28.146891 31869 tablet_bootstrap.cc:492] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f: No bootstrap required, opened a new log
I20240502 14:00:28.146972 31869 ts_tablet_manager.cc:1367] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 14:00:28.147405 31869 raft_consensus.cc:348] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.147493 31869 raft_consensus.cc:374] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.147522 31869 raft_consensus.cc:729] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.147635 31869 consensus_queue.cc:260] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.147833 31872 tablet_bootstrap.cc:654] T 903f5d6c1faf4267b7f7d7dc32670a52 P 14753c93125941acaad7283edc669052: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.147891 31869 ts_tablet_manager.cc:1398] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:28.148108 31869 tablet_bootstrap.cc:492] T be25ac1c6e7242519ff636242718cb79 P ad5befacd63742a1b5b69fdacdb8cf8f: Bootstrap starting.
I20240502 14:00:28.148931 31869 tablet_bootstrap.cc:654] T be25ac1c6e7242519ff636242718cb79 P ad5befacd63742a1b5b69fdacdb8cf8f: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.149531 31872 tablet_bootstrap.cc:492] T 903f5d6c1faf4267b7f7d7dc32670a52 P 14753c93125941acaad7283edc669052: No bootstrap required, opened a new log
I20240502 14:00:28.149619 31872 ts_tablet_manager.cc:1367] T 903f5d6c1faf4267b7f7d7dc32670a52 P 14753c93125941acaad7283edc669052: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:00:28.150068 31872 raft_consensus.cc:348] T 903f5d6c1faf4267b7f7d7dc32670a52 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.150135 31869 tablet_bootstrap.cc:492] T be25ac1c6e7242519ff636242718cb79 P ad5befacd63742a1b5b69fdacdb8cf8f: No bootstrap required, opened a new log
I20240502 14:00:28.150174 31872 raft_consensus.cc:374] T 903f5d6c1faf4267b7f7d7dc32670a52 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.150206 31869 ts_tablet_manager.cc:1367] T be25ac1c6e7242519ff636242718cb79 P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:28.150211 31872 raft_consensus.cc:729] T 903f5d6c1faf4267b7f7d7dc32670a52 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14753c93125941acaad7283edc669052, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.150333 31872 consensus_queue.cc:260] T 903f5d6c1faf4267b7f7d7dc32670a52 P 14753c93125941acaad7283edc669052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.150573 31872 ts_tablet_manager.cc:1398] T 903f5d6c1faf4267b7f7d7dc32670a52 P 14753c93125941acaad7283edc669052: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:28.150565 31869 raft_consensus.cc:348] T be25ac1c6e7242519ff636242718cb79 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.150650 31869 raft_consensus.cc:374] T be25ac1c6e7242519ff636242718cb79 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.150687 31869 raft_consensus.cc:729] T be25ac1c6e7242519ff636242718cb79 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.150784 31869 consensus_queue.cc:260] T be25ac1c6e7242519ff636242718cb79 P ad5befacd63742a1b5b69fdacdb8cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.151001 31869 ts_tablet_manager.cc:1398] T be25ac1c6e7242519ff636242718cb79 P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:28.172618 31885 raft_consensus.cc:482] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:28.172770 31885 raft_consensus.cc:504] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.173642 31885 leader_election.cc:288] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193), ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
I20240502 14:00:28.177412 31135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e02dc2a98d04bddafdd1b933d89b4dd" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc" is_pre_election: true
I20240502 14:00:28.177618 31135 raft_consensus.cc:2454] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 0.
I20240502 14:00:28.177659 30865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e02dc2a98d04bddafdd1b933d89b4dd" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" is_pre_election: true
I20240502 14:00:28.177872 30865 raft_consensus.cc:2454] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 0.
I20240502 14:00:28.178001 30936 leader_election.cc:302] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fab10dcfb8542fca2b965fed6c67929, 8c61042437444f5f92c76109dfb229fc; no voters:
I20240502 14:00:28.178193 31885 raft_consensus.cc:2781] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:28.178260 31885 raft_consensus.cc:482] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:28.178290 31885 raft_consensus.cc:3037] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.179075 31885 raft_consensus.cc:504] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.179358 31885 leader_election.cc:288] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 election: Requested vote from peers 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193), ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
I20240502 14:00:28.179610 31135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e02dc2a98d04bddafdd1b933d89b4dd" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc"
I20240502 14:00:28.179653 30865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e02dc2a98d04bddafdd1b933d89b4dd" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
I20240502 14:00:28.179708 31135 raft_consensus.cc:3037] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.179742 30865 raft_consensus.cc:3037] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.180483 30865 raft_consensus.cc:2454] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 1.
I20240502 14:00:28.180521 31135 raft_consensus.cc:2454] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 1.
I20240502 14:00:28.180728 30933 leader_election.cc:302] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fab10dcfb8542fca2b965fed6c67929, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:28.180869 31885 raft_consensus.cc:2781] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:28.180938 31885 raft_consensus.cc:686] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Becoming Leader. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Running, Role: LEADER
I20240502 14:00:28.181186 31885 consensus_queue.cc:237] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.183712 30712 catalog_manager.cc:5484] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66). New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:28.202374 31182 tablet.cc:2369] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 14:00:28.283195 31050 tablet.cc:2369] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 14:00:28.285749 31880 raft_consensus.cc:482] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:28.285856 31880 raft_consensus.cc:504] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.286803 31880 leader_election.cc:288] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14753c93125941acaad7283edc669052 (127.24.66.68:37029), 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:28.290978 31135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "903f5d6c1faf4267b7f7d7dc32670a52" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc" is_pre_election: true
I20240502 14:00:28.291066 31267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "903f5d6c1faf4267b7f7d7dc32670a52" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052" is_pre_election: true
I20240502 14:00:28.291124 31135 raft_consensus.cc:2454] T 903f5d6c1faf4267b7f7d7dc32670a52 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 0.
I20240502 14:00:28.291316 31267 raft_consensus.cc:2454] T 903f5d6c1faf4267b7f7d7dc32670a52 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 0.
I20240502 14:00:28.291435 30798 leader_election.cc:302] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c61042437444f5f92c76109dfb229fc, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:28.291667 31880 raft_consensus.cc:2781] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:28.291728 31880 raft_consensus.cc:482] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:28.291755 31880 raft_consensus.cc:3037] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.292553 31880 raft_consensus.cc:504] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.292830 31880 leader_election.cc:288] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 election: Requested vote from peers 14753c93125941acaad7283edc669052 (127.24.66.68:37029), 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:28.293126 31267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "903f5d6c1faf4267b7f7d7dc32670a52" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052"
I20240502 14:00:28.293183 31135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "903f5d6c1faf4267b7f7d7dc32670a52" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc"
I20240502 14:00:28.293234 31267 raft_consensus.cc:3037] T 903f5d6c1faf4267b7f7d7dc32670a52 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.293304 31135 raft_consensus.cc:3037] T 903f5d6c1faf4267b7f7d7dc32670a52 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.294214 31267 raft_consensus.cc:2454] T 903f5d6c1faf4267b7f7d7dc32670a52 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 1.
I20240502 14:00:28.294214 31135 raft_consensus.cc:2454] T 903f5d6c1faf4267b7f7d7dc32670a52 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 1.
I20240502 14:00:28.294471 30798 leader_election.cc:302] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c61042437444f5f92c76109dfb229fc, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:28.294639 31880 raft_consensus.cc:2781] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:28.294819 31880 raft_consensus.cc:686] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Becoming Leader. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Running, Role: LEADER
I20240502 14:00:28.295102 31880 consensus_queue.cc:237] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.297740 30712 catalog_manager.cc:5484] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: term changed from 0 to 1, leader changed from <none> to ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65). New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } }
W20240502 14:00:28.344702 30915 tablet.cc:2369] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 14:00:28.362248 31317 tablet.cc:2369] T 903f5d6c1faf4267b7f7d7dc32670a52 P 14753c93125941acaad7283edc669052: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 14:00:28.397986 31885 raft_consensus.cc:482] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:28.398082 31885 raft_consensus.cc:504] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.398532 31885 leader_election.cc:288] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14753c93125941acaad7283edc669052 (127.24.66.68:37029), ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
I20240502 14:00:28.398901 30865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be25ac1c6e7242519ff636242718cb79" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" is_pre_election: true
I20240502 14:00:28.399024 30865 raft_consensus.cc:2454] T be25ac1c6e7242519ff636242718cb79 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 0.
I20240502 14:00:28.399283 30933 leader_election.cc:302] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fab10dcfb8542fca2b965fed6c67929, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:28.399443 31885 raft_consensus.cc:2781] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:28.399507 31885 raft_consensus.cc:482] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:28.399560 31885 raft_consensus.cc:3037] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.400315 31885 raft_consensus.cc:504] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.400650 31885 leader_election.cc:288] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 election: Requested vote from peers 14753c93125941acaad7283edc669052 (127.24.66.68:37029), ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
I20240502 14:00:28.400962 30865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be25ac1c6e7242519ff636242718cb79" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
I20240502 14:00:28.401094 30865 raft_consensus.cc:3037] T be25ac1c6e7242519ff636242718cb79 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.401921 31267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be25ac1c6e7242519ff636242718cb79" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052" is_pre_election: true
I20240502 14:00:28.401978 30865 raft_consensus.cc:2454] T be25ac1c6e7242519ff636242718cb79 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 1.
I20240502 14:00:28.402029 31266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be25ac1c6e7242519ff636242718cb79" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052"
I20240502 14:00:28.402055 31267 raft_consensus.cc:2454] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 0.
I20240502 14:00:28.402119 31266 raft_consensus.cc:3037] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.402213 30933 leader_election.cc:302] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fab10dcfb8542fca2b965fed6c67929, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:28.402360 31885 raft_consensus.cc:2781] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:28.402451 31885 raft_consensus.cc:686] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Becoming Leader. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Running, Role: LEADER
I20240502 14:00:28.402581 31885 consensus_queue.cc:237] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:28.402978 31266 raft_consensus.cc:2454] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 1.
I20240502 14:00:28.404227 30712 catalog_manager.cc:5484] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66). New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:28.507284 31896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:28.507447 31896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:28.510227 31896 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240502 14:00:28.587725 30865 raft_consensus.cc:1261] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:28.588169 31266 raft_consensus.cc:1261] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:28.588580 31878 consensus_queue.cc:1035] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:28.588989 31878 consensus_queue.cc:1035] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:28.590983 30865 raft_consensus.cc:1261] T be25ac1c6e7242519ff636242718cb79 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:28.591380 31885 consensus_queue.cc:1035] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:28.592361 31135 raft_consensus.cc:1261] T 903f5d6c1faf4267b7f7d7dc32670a52 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:28.593143 31880 consensus_queue.cc:1035] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:28.597110 31134 raft_consensus.cc:1261] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:28.597954 31885 consensus_queue.cc:1035] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:28.598650 31926 mvcc.cc:205] Tried to move back new op lower bound from 7023240923505352704 to 7023240922738032640. Current Snapshot: MvccSnapshot[applied={T|T < 7023240923492130816}]
I20240502 14:00:28.601058 31930 mvcc.cc:205] Tried to move back new op lower bound from 7023240923505352704 to 7023240922738032640. Current Snapshot: MvccSnapshot[applied={T|T < 7023240923492130816}]
I20240502 14:00:28.602006 31267 raft_consensus.cc:1261] T 903f5d6c1faf4267b7f7d7dc32670a52 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:28.612375 31932 mvcc.cc:205] Tried to move back new op lower bound from 7023240923492130816 to 7023240922738032640. Current Snapshot: MvccSnapshot[applied={T|T < 7023240923492130816}]
I20240502 14:00:28.615890 31893 consensus_queue.cc:1035] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:28.624394 31936 mvcc.cc:205] Tried to move back new op lower bound from 7023240923505172480 to 7023240922299019264. Current Snapshot: MvccSnapshot[applied={T|T < 7023240923505172480}]
INSERT report
rows total: 2000
time total: 98.6774 ms
time per row: 0.0493387 ms
I20240502 14:00:28.653793 30711 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38902:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:28.658417 31115 tablet_service.cc:1467] Processing CreateTablet for tablet 08588e44069d4044b2d09eb02d252f82 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a4f4cd48a9854811903166c61a300083]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.658494 30981 tablet_service.cc:1467] Processing CreateTablet for tablet 08588e44069d4044b2d09eb02d252f82 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a4f4cd48a9854811903166c61a300083]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.658684 31115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08588e44069d4044b2d09eb02d252f82. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.658685 31246 tablet_service.cc:1467] Processing CreateTablet for tablet 08588e44069d4044b2d09eb02d252f82 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a4f4cd48a9854811903166c61a300083]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.658752 30981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08588e44069d4044b2d09eb02d252f82. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.658891 31246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08588e44069d4044b2d09eb02d252f82. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.660389 31955 tablet_bootstrap.cc:492] T 08588e44069d4044b2d09eb02d252f82 P 1fab10dcfb8542fca2b965fed6c67929: Bootstrap starting.
I20240502 14:00:28.660867 31956 tablet_bootstrap.cc:492] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc: Bootstrap starting.
I20240502 14:00:28.661091 30982 tablet_service.cc:1467] Processing CreateTablet for tablet 42ecb8fd71404d7b9eb25af03d2200af (DEFAULT_TABLE table=rebalance_test_table_1 [id=a4f4cd48a9854811903166c61a300083]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.661304 30982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42ecb8fd71404d7b9eb25af03d2200af. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.661312 31955 tablet_bootstrap.cc:654] T 08588e44069d4044b2d09eb02d252f82 P 1fab10dcfb8542fca2b965fed6c67929: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.661846 31956 tablet_bootstrap.cc:654] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.662367 31955 tablet_bootstrap.cc:492] T 08588e44069d4044b2d09eb02d252f82 P 1fab10dcfb8542fca2b965fed6c67929: No bootstrap required, opened a new log
I20240502 14:00:28.662447 31955 ts_tablet_manager.cc:1367] T 08588e44069d4044b2d09eb02d252f82 P 1fab10dcfb8542fca2b965fed6c67929: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:28.662846 31956 tablet_bootstrap.cc:492] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc: No bootstrap required, opened a new log
I20240502 14:00:28.662894 31955 raft_consensus.cc:348] T 08588e44069d4044b2d09eb02d252f82 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:28.662966 31956 ts_tablet_manager.cc:1367] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:28.662995 31955 raft_consensus.cc:374] T 08588e44069d4044b2d09eb02d252f82 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.663025 31955 raft_consensus.cc:729] T 08588e44069d4044b2d09eb02d252f82 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.663173 30843 tablet_service.cc:1467] Processing CreateTablet for tablet 42ecb8fd71404d7b9eb25af03d2200af (DEFAULT_TABLE table=rebalance_test_table_1 [id=a4f4cd48a9854811903166c61a300083]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.663183 31955 consensus_queue.cc:260] T 08588e44069d4044b2d09eb02d252f82 P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:28.663416 30843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42ecb8fd71404d7b9eb25af03d2200af. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.663447 31955 ts_tablet_manager.cc:1398] T 08588e44069d4044b2d09eb02d252f82 P 1fab10dcfb8542fca2b965fed6c67929: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:28.663452 31956 raft_consensus.cc:348] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:28.663574 31956 raft_consensus.cc:374] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.663640 31956 raft_consensus.cc:729] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.663648 31955 tablet_bootstrap.cc:492] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929: Bootstrap starting.
I20240502 14:00:28.663789 31956 consensus_queue.cc:260] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:28.664158 31956 ts_tablet_manager.cc:1398] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:28.664517 31955 tablet_bootstrap.cc:654] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.665298 31960 tablet_bootstrap.cc:492] T 42ecb8fd71404d7b9eb25af03d2200af P ad5befacd63742a1b5b69fdacdb8cf8f: Bootstrap starting.
I20240502 14:00:28.665366 31955 tablet_bootstrap.cc:492] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929: No bootstrap required, opened a new log
I20240502 14:00:28.665463 31955 ts_tablet_manager.cc:1367] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:28.665791 31955 raft_consensus.cc:348] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:28.665867 31955 raft_consensus.cc:374] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.665901 31955 raft_consensus.cc:729] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.666049 31955 consensus_queue.cc:260] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:28.666152 31960 tablet_bootstrap.cc:654] T 42ecb8fd71404d7b9eb25af03d2200af P ad5befacd63742a1b5b69fdacdb8cf8f: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.666286 31955 ts_tablet_manager.cc:1398] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:28.667135 31115 tablet_service.cc:1467] Processing CreateTablet for tablet 8ec88a191b8d4188985b5e3c91a2a031 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a4f4cd48a9854811903166c61a300083]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.667227 31114 tablet_service.cc:1467] Processing CreateTablet for tablet 42ecb8fd71404d7b9eb25af03d2200af (DEFAULT_TABLE table=rebalance_test_table_1 [id=a4f4cd48a9854811903166c61a300083]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.667344 31115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ec88a191b8d4188985b5e3c91a2a031. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.667568 31114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42ecb8fd71404d7b9eb25af03d2200af. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.668994 30981 tablet_service.cc:1467] Processing CreateTablet for tablet 8ec88a191b8d4188985b5e3c91a2a031 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a4f4cd48a9854811903166c61a300083]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.669188 30981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ec88a191b8d4188985b5e3c91a2a031. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.669324 31245 tablet_service.cc:1467] Processing CreateTablet for tablet 8ec88a191b8d4188985b5e3c91a2a031 (DEFAULT_TABLE table=rebalance_test_table_1 [id=a4f4cd48a9854811903166c61a300083]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:28.669538 31245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ec88a191b8d4188985b5e3c91a2a031. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:28.669857 31956 tablet_bootstrap.cc:492] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc: Bootstrap starting.
I20240502 14:00:28.670593 31956 tablet_bootstrap.cc:654] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.670882 31955 tablet_bootstrap.cc:492] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929: Bootstrap starting.
I20240502 14:00:28.671026 31962 tablet_bootstrap.cc:492] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052: Bootstrap starting.
I20240502 14:00:28.671818 31962 tablet_bootstrap.cc:654] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.671895 31955 tablet_bootstrap.cc:654] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.672279 31956 tablet_bootstrap.cc:492] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc: No bootstrap required, opened a new log
I20240502 14:00:28.672444 31956 ts_tablet_manager.cc:1367] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:28.672740 31960 tablet_bootstrap.cc:492] T 42ecb8fd71404d7b9eb25af03d2200af P ad5befacd63742a1b5b69fdacdb8cf8f: No bootstrap required, opened a new log
I20240502 14:00:28.672830 31960 ts_tablet_manager.cc:1367] T 42ecb8fd71404d7b9eb25af03d2200af P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240502 14:00:28.672834 31956 raft_consensus.cc:348] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:28.672945 31956 raft_consensus.cc:374] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.672964 31955 tablet_bootstrap.cc:492] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929: No bootstrap required, opened a new log
I20240502 14:00:28.673023 31955 ts_tablet_manager.cc:1367] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:28.673017 31956 raft_consensus.cc:729] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.673151 31956 consensus_queue.cc:260] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:28.673247 31960 raft_consensus.cc:348] T 42ecb8fd71404d7b9eb25af03d2200af P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:28.673342 31960 raft_consensus.cc:374] T 42ecb8fd71404d7b9eb25af03d2200af P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.673377 31960 raft_consensus.cc:729] T 42ecb8fd71404d7b9eb25af03d2200af P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.673377 31955 raft_consensus.cc:348] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:28.673455 31956 ts_tablet_manager.cc:1398] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:28.673476 31955 raft_consensus.cc:374] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.673512 31955 raft_consensus.cc:729] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.673487 31960 consensus_queue.cc:260] T 42ecb8fd71404d7b9eb25af03d2200af P ad5befacd63742a1b5b69fdacdb8cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:28.673594 31956 tablet_bootstrap.cc:492] T 42ecb8fd71404d7b9eb25af03d2200af P 8c61042437444f5f92c76109dfb229fc: Bootstrap starting.
I20240502 14:00:28.673607 31955 consensus_queue.cc:260] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:28.673743 31960 ts_tablet_manager.cc:1398] T 42ecb8fd71404d7b9eb25af03d2200af P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:28.673852 31955 ts_tablet_manager.cc:1398] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:28.674340 31962 tablet_bootstrap.cc:492] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052: No bootstrap required, opened a new log
I20240502 14:00:28.674417 31962 ts_tablet_manager.cc:1367] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:28.674743 31962 raft_consensus.cc:348] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:28.674821 31962 raft_consensus.cc:374] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.674842 31962 raft_consensus.cc:729] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14753c93125941acaad7283edc669052, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.674968 31962 consensus_queue.cc:260] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:28.675595 31962 ts_tablet_manager.cc:1398] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:28.675786 31962 tablet_bootstrap.cc:492] T 8ec88a191b8d4188985b5e3c91a2a031 P 14753c93125941acaad7283edc669052: Bootstrap starting.
I20240502 14:00:28.676609 31962 tablet_bootstrap.cc:654] T 8ec88a191b8d4188985b5e3c91a2a031 P 14753c93125941acaad7283edc669052: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.677748 31962 tablet_bootstrap.cc:492] T 8ec88a191b8d4188985b5e3c91a2a031 P 14753c93125941acaad7283edc669052: No bootstrap required, opened a new log
I20240502 14:00:28.678128 31962 ts_tablet_manager.cc:1367] T 8ec88a191b8d4188985b5e3c91a2a031 P 14753c93125941acaad7283edc669052: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:28.678505 31962 raft_consensus.cc:348] T 8ec88a191b8d4188985b5e3c91a2a031 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:28.678613 31962 raft_consensus.cc:374] T 8ec88a191b8d4188985b5e3c91a2a031 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.678651 31962 raft_consensus.cc:729] T 8ec88a191b8d4188985b5e3c91a2a031 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14753c93125941acaad7283edc669052, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.678812 31962 consensus_queue.cc:260] T 8ec88a191b8d4188985b5e3c91a2a031 P 14753c93125941acaad7283edc669052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:28.679337 31962 ts_tablet_manager.cc:1398] T 8ec88a191b8d4188985b5e3c91a2a031 P 14753c93125941acaad7283edc669052: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:28.679567 31956 tablet_bootstrap.cc:654] T 42ecb8fd71404d7b9eb25af03d2200af P 8c61042437444f5f92c76109dfb229fc: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:28.680508 31956 tablet_bootstrap.cc:492] T 42ecb8fd71404d7b9eb25af03d2200af P 8c61042437444f5f92c76109dfb229fc: No bootstrap required, opened a new log
I20240502 14:00:28.680588 31956 ts_tablet_manager.cc:1367] T 42ecb8fd71404d7b9eb25af03d2200af P 8c61042437444f5f92c76109dfb229fc: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:00:28.680912 31956 raft_consensus.cc:348] T 42ecb8fd71404d7b9eb25af03d2200af P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:28.680992 31956 raft_consensus.cc:374] T 42ecb8fd71404d7b9eb25af03d2200af P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:28.681012 31956 raft_consensus.cc:729] T 42ecb8fd71404d7b9eb25af03d2200af P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Initialized, Role: FOLLOWER
I20240502 14:00:28.681104 31956 consensus_queue.cc:260] T 42ecb8fd71404d7b9eb25af03d2200af P 8c61042437444f5f92c76109dfb229fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:28.681360 31956 ts_tablet_manager.cc:1398] T 42ecb8fd71404d7b9eb25af03d2200af P 8c61042437444f5f92c76109dfb229fc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:28.688269 31885 raft_consensus.cc:482] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:28.688371 31885 raft_consensus.cc:504] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:28.688652 31885 leader_election.cc:288] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553), 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:28.688925 31134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42ecb8fd71404d7b9eb25af03d2200af" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc" is_pre_election: true
I20240502 14:00:28.688938 30864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42ecb8fd71404d7b9eb25af03d2200af" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" is_pre_election: true
I20240502 14:00:28.689033 31134 raft_consensus.cc:2454] T 42ecb8fd71404d7b9eb25af03d2200af P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 0.
I20240502 14:00:28.689033 30864 raft_consensus.cc:2454] T 42ecb8fd71404d7b9eb25af03d2200af P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 0.
I20240502 14:00:28.689241 30936 leader_election.cc:302] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fab10dcfb8542fca2b965fed6c67929, 8c61042437444f5f92c76109dfb229fc; no voters:
I20240502 14:00:28.689379 31885 raft_consensus.cc:2781] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:28.689435 31885 raft_consensus.cc:482] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:28.689460 31885 raft_consensus.cc:3037] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.690037 31885 raft_consensus.cc:504] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:28.690299 31885 leader_election.cc:288] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 election: Requested vote from peers ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553), 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:28.690554 30864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42ecb8fd71404d7b9eb25af03d2200af" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
I20240502 14:00:28.690567 31134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42ecb8fd71404d7b9eb25af03d2200af" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc"
I20240502 14:00:28.690635 31134 raft_consensus.cc:3037] T 42ecb8fd71404d7b9eb25af03d2200af P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.690641 30864 raft_consensus.cc:3037] T 42ecb8fd71404d7b9eb25af03d2200af P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.691244 31134 raft_consensus.cc:2454] T 42ecb8fd71404d7b9eb25af03d2200af P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 1.
I20240502 14:00:28.691249 30864 raft_consensus.cc:2454] T 42ecb8fd71404d7b9eb25af03d2200af P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 1.
I20240502 14:00:28.691475 30933 leader_election.cc:302] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fab10dcfb8542fca2b965fed6c67929, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:28.691658 31885 raft_consensus.cc:2781] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:28.691732 31885 raft_consensus.cc:686] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Becoming Leader. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Running, Role: LEADER
I20240502 14:00:28.691839 31885 consensus_queue.cc:237] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:28.693061 30709 catalog_manager.cc:5484] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66). New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:28.861042 31885 raft_consensus.cc:482] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:28.861145 31885 raft_consensus.cc:504] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:28.861493 31885 leader_election.cc:288] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193), 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:28.861877 31134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ec88a191b8d4188985b5e3c91a2a031" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc" is_pre_election: true
I20240502 14:00:28.861915 31267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ec88a191b8d4188985b5e3c91a2a031" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052" is_pre_election: true
I20240502 14:00:28.862020 31134 raft_consensus.cc:2454] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 0.
I20240502 14:00:28.862030 31267 raft_consensus.cc:2454] T 8ec88a191b8d4188985b5e3c91a2a031 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 0.
I20240502 14:00:28.862267 30936 leader_election.cc:302] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fab10dcfb8542fca2b965fed6c67929, 8c61042437444f5f92c76109dfb229fc; no voters:
I20240502 14:00:28.862421 31885 raft_consensus.cc:2781] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:28.862475 31885 raft_consensus.cc:482] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:28.862516 31885 raft_consensus.cc:3037] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.863189 31885 raft_consensus.cc:504] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:28.863492 31885 leader_election.cc:288] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 election: Requested vote from peers 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193), 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:28.863812 31134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ec88a191b8d4188985b5e3c91a2a031" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc"
I20240502 14:00:28.863833 31267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ec88a191b8d4188985b5e3c91a2a031" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052"
I20240502 14:00:28.863924 31267 raft_consensus.cc:3037] T 8ec88a191b8d4188985b5e3c91a2a031 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.863924 31134 raft_consensus.cc:3037] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.864847 31134 raft_consensus.cc:2454] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 1.
I20240502 14:00:28.864847 31267 raft_consensus.cc:2454] T 8ec88a191b8d4188985b5e3c91a2a031 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 1.
I20240502 14:00:28.865118 30936 leader_election.cc:302] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 1fab10dcfb8542fca2b965fed6c67929; no voters:
I20240502 14:00:28.865264 31885 raft_consensus.cc:2781] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:28.865350 31885 raft_consensus.cc:686] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Becoming Leader. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Running, Role: LEADER
I20240502 14:00:28.865525 31885 consensus_queue.cc:237] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:28.866866 30709 catalog_manager.cc:5484] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66). New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:28.985991 31879 raft_consensus.cc:482] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:28.986086 31879 raft_consensus.cc:504] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:28.986990 31879 leader_election.cc:288] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193), 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:28.991382 31134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08588e44069d4044b2d09eb02d252f82" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc" is_pre_election: true
I20240502 14:00:28.991444 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08588e44069d4044b2d09eb02d252f82" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929" is_pre_election: true
I20240502 14:00:28.991498 31134 raft_consensus.cc:2454] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 0.
I20240502 14:00:28.991650 31003 raft_consensus.cc:2454] T 08588e44069d4044b2d09eb02d252f82 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 0.
I20240502 14:00:28.991815 31202 leader_election.cc:302] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 8c61042437444f5f92c76109dfb229fc; no voters:
I20240502 14:00:28.992038 31879 raft_consensus.cc:2781] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:28.992094 31879 raft_consensus.cc:482] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:28.992118 31879 raft_consensus.cc:3037] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.992944 31879 raft_consensus.cc:504] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:28.993227 31879 leader_election.cc:288] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 election: Requested vote from peers 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193), 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:28.993534 31134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08588e44069d4044b2d09eb02d252f82" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc"
I20240502 14:00:28.993571 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08588e44069d4044b2d09eb02d252f82" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
I20240502 14:00:28.993672 31003 raft_consensus.cc:3037] T 08588e44069d4044b2d09eb02d252f82 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.993654 31134 raft_consensus.cc:3037] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:28.994361 31134 raft_consensus.cc:2454] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 1.
I20240502 14:00:28.994364 31003 raft_consensus.cc:2454] T 08588e44069d4044b2d09eb02d252f82 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 1.
I20240502 14:00:28.994604 31202 leader_election.cc:302] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 8c61042437444f5f92c76109dfb229fc; no voters:
I20240502 14:00:28.994782 31879 raft_consensus.cc:2781] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:28.994961 31879 raft_consensus.cc:686] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Becoming Leader. State: Replica: 14753c93125941acaad7283edc669052, State: Running, Role: LEADER
I20240502 14:00:28.995226 31879 consensus_queue.cc:237] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:28.997452 30709 catalog_manager.cc:5484] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14753c93125941acaad7283edc669052 (127.24.66.68). New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } }
W20240502 14:00:29.126318 31975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:29.126523 31975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:29.129338 31975 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240502 14:00:29.170019 31885 consensus_queue.cc:1035] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:29.174041 31929 consensus_queue.cc:1035] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:29.204931 31267 raft_consensus.cc:1261] T 8ec88a191b8d4188985b5e3c91a2a031 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:29.208690 31003 raft_consensus.cc:1261] T 08588e44069d4044b2d09eb02d252f82 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:29.208918 31134 raft_consensus.cc:1261] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:29.209091 31929 consensus_queue.cc:1035] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:29.209623 31879 consensus_queue.cc:1035] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:29.210110 31973 consensus_queue.cc:1035] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:29.217531 31133 raft_consensus.cc:1261] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:29.219125 31929 consensus_queue.cc:1035] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 99.3856 ms
time per row: 0.0496928 ms
I20240502 14:00:29.265879 30710 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38902:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:29.270632 31245 tablet_service.cc:1467] Processing CreateTablet for tablet aaca0594b68146cdbd18946f123693be (DEFAULT_TABLE table=rebalance_test_table_2 [id=2109014949af4d09893b47c9b10ba7b5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.270916 31245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aaca0594b68146cdbd18946f123693be. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.271695 30843 tablet_service.cc:1467] Processing CreateTablet for tablet aaca0594b68146cdbd18946f123693be (DEFAULT_TABLE table=rebalance_test_table_2 [id=2109014949af4d09893b47c9b10ba7b5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.271692 30982 tablet_service.cc:1467] Processing CreateTablet for tablet 2051076a81274b688554c218c107ee1c (DEFAULT_TABLE table=rebalance_test_table_2 [id=2109014949af4d09893b47c9b10ba7b5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.271929 30982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2051076a81274b688554c218c107ee1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.271929 30843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aaca0594b68146cdbd18946f123693be. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.272692 32037 tablet_bootstrap.cc:492] T aaca0594b68146cdbd18946f123693be P 14753c93125941acaad7283edc669052: Bootstrap starting.
I20240502 14:00:29.273079 31114 tablet_service.cc:1467] Processing CreateTablet for tablet 03fb6c61ce4c4ccdafdc9ca7a40db4f5 (DEFAULT_TABLE table=rebalance_test_table_2 [id=2109014949af4d09893b47c9b10ba7b5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.273326 31114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03fb6c61ce4c4ccdafdc9ca7a40db4f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.273629 32037 tablet_bootstrap.cc:654] T aaca0594b68146cdbd18946f123693be P 14753c93125941acaad7283edc669052: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.274072 30845 tablet_service.cc:1467] Processing CreateTablet for tablet 03fb6c61ce4c4ccdafdc9ca7a40db4f5 (DEFAULT_TABLE table=rebalance_test_table_2 [id=2109014949af4d09893b47c9b10ba7b5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.274308 30845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03fb6c61ce4c4ccdafdc9ca7a40db4f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.274372 32039 tablet_bootstrap.cc:492] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f: Bootstrap starting.
I20240502 14:00:29.275146 32039 tablet_bootstrap.cc:654] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.275219 32041 tablet_bootstrap.cc:492] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc: Bootstrap starting.
I20240502 14:00:29.275991 32041 tablet_bootstrap.cc:654] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.276136 32039 tablet_bootstrap.cc:492] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f: No bootstrap required, opened a new log
I20240502 14:00:29.276173 31246 tablet_service.cc:1467] Processing CreateTablet for tablet 03fb6c61ce4c4ccdafdc9ca7a40db4f5 (DEFAULT_TABLE table=rebalance_test_table_2 [id=2109014949af4d09893b47c9b10ba7b5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.276223 32039 ts_tablet_manager.cc:1367] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:29.276527 30842 tablet_service.cc:1467] Processing CreateTablet for tablet 2051076a81274b688554c218c107ee1c (DEFAULT_TABLE table=rebalance_test_table_2 [id=2109014949af4d09893b47c9b10ba7b5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.276625 32039 raft_consensus.cc:348] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:29.276717 32039 raft_consensus.cc:374] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.276742 30842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2051076a81274b688554c218c107ee1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.276748 32039 raft_consensus.cc:729] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.276886 30981 tablet_service.cc:1467] Processing CreateTablet for tablet aaca0594b68146cdbd18946f123693be (DEFAULT_TABLE table=rebalance_test_table_2 [id=2109014949af4d09893b47c9b10ba7b5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.276995 32039 consensus_queue.cc:260] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:29.277102 30981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aaca0594b68146cdbd18946f123693be. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.277127 32041 tablet_bootstrap.cc:492] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc: No bootstrap required, opened a new log
I20240502 14:00:29.277189 32041 ts_tablet_manager.cc:1367] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:29.277629 32041 raft_consensus.cc:348] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:29.277715 32041 raft_consensus.cc:374] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.277747 32041 raft_consensus.cc:729] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.277884 32041 consensus_queue.cc:260] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:29.277973 32039 ts_tablet_manager.cc:1398] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:29.278113 32039 tablet_bootstrap.cc:492] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f: Bootstrap starting.
I20240502 14:00:29.278823 31246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03fb6c61ce4c4ccdafdc9ca7a40db4f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.279201 32039 tablet_bootstrap.cc:654] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.280144 32038 tablet_bootstrap.cc:492] T 2051076a81274b688554c218c107ee1c P 1fab10dcfb8542fca2b965fed6c67929: Bootstrap starting.
I20240502 14:00:29.280903 32038 tablet_bootstrap.cc:654] T 2051076a81274b688554c218c107ee1c P 1fab10dcfb8542fca2b965fed6c67929: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.281390 32039 tablet_bootstrap.cc:492] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f: No bootstrap required, opened a new log
I20240502 14:00:29.281466 32039 ts_tablet_manager.cc:1367] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:29.281908 32037 tablet_bootstrap.cc:492] T aaca0594b68146cdbd18946f123693be P 14753c93125941acaad7283edc669052: No bootstrap required, opened a new log
I20240502 14:00:29.281855 32039 raft_consensus.cc:348] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:29.281940 32039 raft_consensus.cc:374] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.281968 32039 raft_consensus.cc:729] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.281978 32037 ts_tablet_manager.cc:1367] T aaca0594b68146cdbd18946f123693be P 14753c93125941acaad7283edc669052: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 14:00:29.282083 32039 consensus_queue.cc:260] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:29.282189 31246 tablet_service.cc:1467] Processing CreateTablet for tablet 2051076a81274b688554c218c107ee1c (DEFAULT_TABLE table=rebalance_test_table_2 [id=2109014949af4d09893b47c9b10ba7b5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.282305 32039 ts_tablet_manager.cc:1398] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:29.282374 31246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2051076a81274b688554c218c107ee1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.282444 32039 tablet_bootstrap.cc:492] T 2051076a81274b688554c218c107ee1c P ad5befacd63742a1b5b69fdacdb8cf8f: Bootstrap starting.
I20240502 14:00:29.283166 32039 tablet_bootstrap.cc:654] T 2051076a81274b688554c218c107ee1c P ad5befacd63742a1b5b69fdacdb8cf8f: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.282380 32037 raft_consensus.cc:348] T aaca0594b68146cdbd18946f123693be P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:29.283780 32037 raft_consensus.cc:374] T aaca0594b68146cdbd18946f123693be P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.283813 32037 raft_consensus.cc:729] T aaca0594b68146cdbd18946f123693be P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14753c93125941acaad7283edc669052, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.283934 32037 consensus_queue.cc:260] T aaca0594b68146cdbd18946f123693be P 14753c93125941acaad7283edc669052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:29.284039 32041 ts_tablet_manager.cc:1398] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20240502 14:00:29.284198 32037 ts_tablet_manager.cc:1398] T aaca0594b68146cdbd18946f123693be P 14753c93125941acaad7283edc669052: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:29.284704 32038 tablet_bootstrap.cc:492] T 2051076a81274b688554c218c107ee1c P 1fab10dcfb8542fca2b965fed6c67929: No bootstrap required, opened a new log
I20240502 14:00:29.284777 32038 ts_tablet_manager.cc:1367] T 2051076a81274b688554c218c107ee1c P 1fab10dcfb8542fca2b965fed6c67929: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:29.285005 32037 tablet_bootstrap.cc:492] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 14753c93125941acaad7283edc669052: Bootstrap starting.
I20240502 14:00:29.285156 32038 raft_consensus.cc:348] T 2051076a81274b688554c218c107ee1c P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:29.285250 32038 raft_consensus.cc:374] T 2051076a81274b688554c218c107ee1c P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.285281 32038 raft_consensus.cc:729] T 2051076a81274b688554c218c107ee1c P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.285403 32038 consensus_queue.cc:260] T 2051076a81274b688554c218c107ee1c P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:29.285668 32038 ts_tablet_manager.cc:1398] T 2051076a81274b688554c218c107ee1c P 1fab10dcfb8542fca2b965fed6c67929: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:29.285821 32038 tablet_bootstrap.cc:492] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929: Bootstrap starting.
I20240502 14:00:29.285840 32037 tablet_bootstrap.cc:654] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 14753c93125941acaad7283edc669052: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.286537 32038 tablet_bootstrap.cc:654] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.287134 32039 tablet_bootstrap.cc:492] T 2051076a81274b688554c218c107ee1c P ad5befacd63742a1b5b69fdacdb8cf8f: No bootstrap required, opened a new log
I20240502 14:00:29.287216 32039 ts_tablet_manager.cc:1367] T 2051076a81274b688554c218c107ee1c P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 14:00:29.287688 32039 raft_consensus.cc:348] T 2051076a81274b688554c218c107ee1c P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:29.287771 32039 raft_consensus.cc:374] T 2051076a81274b688554c218c107ee1c P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.287799 32039 raft_consensus.cc:729] T 2051076a81274b688554c218c107ee1c P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.287895 32039 consensus_queue.cc:260] T 2051076a81274b688554c218c107ee1c P ad5befacd63742a1b5b69fdacdb8cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:29.287959 32037 tablet_bootstrap.cc:492] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 14753c93125941acaad7283edc669052: No bootstrap required, opened a new log
I20240502 14:00:29.288019 32037 ts_tablet_manager.cc:1367] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 14753c93125941acaad7283edc669052: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:29.288144 32039 ts_tablet_manager.cc:1398] T 2051076a81274b688554c218c107ee1c P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:29.288430 32037 raft_consensus.cc:348] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:29.288522 32037 raft_consensus.cc:374] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.288576 32037 raft_consensus.cc:729] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14753c93125941acaad7283edc669052, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.288666 32037 consensus_queue.cc:260] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 14753c93125941acaad7283edc669052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:29.288877 32037 ts_tablet_manager.cc:1398] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 14753c93125941acaad7283edc669052: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:29.289024 32037 tablet_bootstrap.cc:492] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052: Bootstrap starting.
I20240502 14:00:29.289876 32037 tablet_bootstrap.cc:654] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.290019 32038 tablet_bootstrap.cc:492] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929: No bootstrap required, opened a new log
I20240502 14:00:29.290084 32038 ts_tablet_manager.cc:1367] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:00:29.290547 32038 raft_consensus.cc:348] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:29.290639 32038 raft_consensus.cc:374] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.290685 32038 raft_consensus.cc:729] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.290794 32038 consensus_queue.cc:260] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:29.291118 32038 ts_tablet_manager.cc:1398] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:29.291275 32037 tablet_bootstrap.cc:492] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052: No bootstrap required, opened a new log
I20240502 14:00:29.291340 32037 ts_tablet_manager.cc:1367] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:29.291743 32037 raft_consensus.cc:348] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:29.291828 32037 raft_consensus.cc:374] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.291858 32037 raft_consensus.cc:729] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14753c93125941acaad7283edc669052, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.291955 32037 consensus_queue.cc:260] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:29.292186 32037 ts_tablet_manager.cc:1398] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:29.451779 31877 raft_consensus.cc:482] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:29.451874 31877 raft_consensus.cc:504] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:29.452899 31877 leader_election.cc:288] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553), 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:29.456979 31267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03fb6c61ce4c4ccdafdc9ca7a40db4f5" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052" is_pre_election: true
I20240502 14:00:29.457105 31267 raft_consensus.cc:2454] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 0.
I20240502 14:00:29.457208 30864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03fb6c61ce4c4ccdafdc9ca7a40db4f5" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" is_pre_election: true
I20240502 14:00:29.457367 30864 raft_consensus.cc:2454] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 0.
I20240502 14:00:29.457454 31070 leader_election.cc:302] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 8c61042437444f5f92c76109dfb229fc; no voters:
I20240502 14:00:29.457679 31877 raft_consensus.cc:2781] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:29.457734 31877 raft_consensus.cc:482] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:29.457784 31877 raft_consensus.cc:3037] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.458546 31877 raft_consensus.cc:504] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:29.458794 31877 leader_election.cc:288] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 1 election: Requested vote from peers ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553), 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:29.459084 30864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03fb6c61ce4c4ccdafdc9ca7a40db4f5" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
I20240502 14:00:29.459146 31267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03fb6c61ce4c4ccdafdc9ca7a40db4f5" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052"
I20240502 14:00:29.459178 30864 raft_consensus.cc:3037] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.459249 31267 raft_consensus.cc:3037] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.459882 30864 raft_consensus.cc:2454] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 1.
I20240502 14:00:29.460093 31267 raft_consensus.cc:2454] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 1.
I20240502 14:00:29.460150 31067 leader_election.cc:302] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c61042437444f5f92c76109dfb229fc, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:29.460410 31877 raft_consensus.cc:2781] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:29.460590 31877 raft_consensus.cc:686] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Becoming Leader. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Running, Role: LEADER
I20240502 14:00:29.460860 31877 consensus_queue.cc:237] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:29.462893 30710 catalog_manager.cc:5484] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c61042437444f5f92c76109dfb229fc (127.24.66.67). New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:29.576958 32016 raft_consensus.cc:482] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:29.577049 32016 raft_consensus.cc:504] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:29.577466 32016 leader_election.cc:288] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395), ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
I20240502 14:00:29.577833 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2051076a81274b688554c218c107ee1c" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929" is_pre_election: true
I20240502 14:00:29.577988 31003 raft_consensus.cc:2454] T 2051076a81274b688554c218c107ee1c P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 0.
I20240502 14:00:29.578279 31201 leader_election.cc:302] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 1fab10dcfb8542fca2b965fed6c67929; no voters:
I20240502 14:00:29.578478 32016 raft_consensus.cc:2781] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:29.578531 32016 raft_consensus.cc:482] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:29.578572 32016 raft_consensus.cc:3037] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.579347 32016 raft_consensus.cc:504] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:29.579644 32016 leader_election.cc:288] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 election: Requested vote from peers 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395), ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
I20240502 14:00:29.579900 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2051076a81274b688554c218c107ee1c" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
I20240502 14:00:29.579998 31003 raft_consensus.cc:3037] T 2051076a81274b688554c218c107ee1c P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.580829 31003 raft_consensus.cc:2454] T 2051076a81274b688554c218c107ee1c P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 1.
I20240502 14:00:29.581048 30864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2051076a81274b688554c218c107ee1c" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" is_pre_election: true
I20240502 14:00:29.581092 31201 leader_election.cc:302] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 1fab10dcfb8542fca2b965fed6c67929; no voters:
I20240502 14:00:29.581122 30865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2051076a81274b688554c218c107ee1c" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
I20240502 14:00:29.581172 30864 raft_consensus.cc:2454] T 2051076a81274b688554c218c107ee1c P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 0.
I20240502 14:00:29.581236 32016 raft_consensus.cc:2781] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:29.581337 32016 raft_consensus.cc:686] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Becoming Leader. State: Replica: 14753c93125941acaad7283edc669052, State: Running, Role: LEADER
I20240502 14:00:29.581480 32016 consensus_queue.cc:237] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:29.582796 30710 catalog_manager.cc:5484] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14753c93125941acaad7283edc669052 (127.24.66.68). New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:29.595054 32004 raft_consensus.cc:482] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:29.595147 32004 raft_consensus.cc:504] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:29.595517 32004 leader_election.cc:288] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14753c93125941acaad7283edc669052 (127.24.66.68:37029), 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:29.595819 31267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aaca0594b68146cdbd18946f123693be" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052" is_pre_election: true
I20240502 14:00:29.595930 31267 raft_consensus.cc:2454] T aaca0594b68146cdbd18946f123693be P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 0.
I20240502 14:00:29.596180 30798 leader_election.cc:302] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:29.596359 32004 raft_consensus.cc:2781] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:29.596406 32004 raft_consensus.cc:482] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:29.596446 32004 raft_consensus.cc:3037] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.597209 32004 raft_consensus.cc:504] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:29.597544 32004 leader_election.cc:288] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 election: Requested vote from peers 14753c93125941acaad7283edc669052 (127.24.66.68:37029), 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:29.597836 31267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aaca0594b68146cdbd18946f123693be" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052"
I20240502 14:00:29.597923 31267 raft_consensus.cc:3037] T aaca0594b68146cdbd18946f123693be P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.598723 31267 raft_consensus.cc:2454] T aaca0594b68146cdbd18946f123693be P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 1.
I20240502 14:00:29.598970 30798 leader_election.cc:302] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:29.598985 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aaca0594b68146cdbd18946f123693be" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929" is_pre_election: true
I20240502 14:00:29.599066 31002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aaca0594b68146cdbd18946f123693be" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
I20240502 14:00:29.599129 32004 raft_consensus.cc:2781] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:29.599083 31003 raft_consensus.cc:2454] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 0.
I20240502 14:00:29.599195 32004 raft_consensus.cc:686] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Becoming Leader. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Running, Role: LEADER
I20240502 14:00:29.599323 32004 consensus_queue.cc:237] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:29.600912 30710 catalog_manager.cc:5484] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: term changed from 0 to 1, leader changed from <none> to ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65). New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:29.737169 32059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:29.737391 32059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:29.740134 32059 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240502 14:00:29.821183 31002 raft_consensus.cc:3037] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.822249 31002 raft_consensus.cc:1261] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:29.822590 32004 consensus_queue.cc:1035] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:29.823032 31267 raft_consensus.cc:1261] T aaca0594b68146cdbd18946f123693be P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:29.823578 32004 consensus_queue.cc:1035] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:29.824030 31267 raft_consensus.cc:1261] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:29.824488 31877 consensus_queue.cc:1035] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:29.824687 30864 raft_consensus.cc:1261] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:29.825048 31877 consensus_queue.cc:1035] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:29.829425 30865 raft_consensus.cc:3037] T 2051076a81274b688554c218c107ee1c P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:29.830380 30865 raft_consensus.cc:1261] T 2051076a81274b688554c218c107ee1c P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:29.830750 32016 consensus_queue.cc:1035] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:29.832479 31003 raft_consensus.cc:1261] T 2051076a81274b688554c218c107ee1c P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:29.832855 32016 consensus_queue.cc:1035] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 112.96 ms
time per row: 0.0564802 ms
I20240502 14:00:29.888597 30712 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38902:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:29.893244 30842 tablet_service.cc:1467] Processing CreateTablet for tablet 5702766b82f24e96a4cb5da5f051fc3f (DEFAULT_TABLE table=rebalance_test_table_3 [id=3fb9e83c8b7c4d90a375d14e792f650d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.893513 30842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5702766b82f24e96a4cb5da5f051fc3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.893486 31115 tablet_service.cc:1467] Processing CreateTablet for tablet f571119d6f624fe88bf1effeaf466085 (DEFAULT_TABLE table=rebalance_test_table_3 [id=3fb9e83c8b7c4d90a375d14e792f650d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.893746 31115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f571119d6f624fe88bf1effeaf466085. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.893827 31245 tablet_service.cc:1467] Processing CreateTablet for tablet 8c77a03ce6a142debba75e0d387aa8ec (DEFAULT_TABLE table=rebalance_test_table_3 [id=3fb9e83c8b7c4d90a375d14e792f650d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.893896 31114 tablet_service.cc:1467] Processing CreateTablet for tablet 5702766b82f24e96a4cb5da5f051fc3f (DEFAULT_TABLE table=rebalance_test_table_3 [id=3fb9e83c8b7c4d90a375d14e792f650d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.894038 31245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c77a03ce6a142debba75e0d387aa8ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.894090 31114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5702766b82f24e96a4cb5da5f051fc3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.895886 30981 tablet_service.cc:1467] Processing CreateTablet for tablet 5702766b82f24e96a4cb5da5f051fc3f (DEFAULT_TABLE table=rebalance_test_table_3 [id=3fb9e83c8b7c4d90a375d14e792f650d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.896133 30981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5702766b82f24e96a4cb5da5f051fc3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.896167 32124 tablet_bootstrap.cc:492] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052: Bootstrap starting.
I20240502 14:00:29.896569 31246 tablet_service.cc:1467] Processing CreateTablet for tablet f571119d6f624fe88bf1effeaf466085 (DEFAULT_TABLE table=rebalance_test_table_3 [id=3fb9e83c8b7c4d90a375d14e792f650d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.896766 31246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f571119d6f624fe88bf1effeaf466085. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.896756 32123 tablet_bootstrap.cc:492] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc: Bootstrap starting.
I20240502 14:00:29.897044 32124 tablet_bootstrap.cc:654] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.897614 32123 tablet_bootstrap.cc:654] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.898087 32124 tablet_bootstrap.cc:492] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052: No bootstrap required, opened a new log
I20240502 14:00:29.898164 32124 ts_tablet_manager.cc:1367] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:29.898578 32124 raft_consensus.cc:348] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:29.898682 32124 raft_consensus.cc:374] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.898715 32124 raft_consensus.cc:729] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14753c93125941acaad7283edc669052, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.898725 32123 tablet_bootstrap.cc:492] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc: No bootstrap required, opened a new log
I20240502 14:00:29.898794 32123 ts_tablet_manager.cc:1367] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:29.898844 32124 consensus_queue.cc:260] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:29.899088 32124 ts_tablet_manager.cc:1398] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:29.899247 32124 tablet_bootstrap.cc:492] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052: Bootstrap starting.
I20240502 14:00:29.899206 32123 raft_consensus.cc:348] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:29.899298 32123 raft_consensus.cc:374] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.899333 32123 raft_consensus.cc:729] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.899415 30981 tablet_service.cc:1467] Processing CreateTablet for tablet 8c77a03ce6a142debba75e0d387aa8ec (DEFAULT_TABLE table=rebalance_test_table_3 [id=3fb9e83c8b7c4d90a375d14e792f650d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.899448 32123 consensus_queue.cc:260] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:29.899623 30981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c77a03ce6a142debba75e0d387aa8ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.899717 32123 ts_tablet_manager.cc:1398] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:29.899910 32123 tablet_bootstrap.cc:492] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc: Bootstrap starting.
I20240502 14:00:29.900024 32124 tablet_bootstrap.cc:654] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.900061 31114 tablet_service.cc:1467] Processing CreateTablet for tablet 8c77a03ce6a142debba75e0d387aa8ec (DEFAULT_TABLE table=rebalance_test_table_3 [id=3fb9e83c8b7c4d90a375d14e792f650d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.900243 31114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c77a03ce6a142debba75e0d387aa8ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.900704 32123 tablet_bootstrap.cc:654] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.901151 32124 tablet_bootstrap.cc:492] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052: No bootstrap required, opened a new log
I20240502 14:00:29.901225 32124 ts_tablet_manager.cc:1367] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 14:00:29.901636 30842 tablet_service.cc:1467] Processing CreateTablet for tablet f571119d6f624fe88bf1effeaf466085 (DEFAULT_TABLE table=rebalance_test_table_3 [id=3fb9e83c8b7c4d90a375d14e792f650d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:29.901793 32124 raft_consensus.cc:348] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:29.901858 30842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f571119d6f624fe88bf1effeaf466085. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:29.901876 32124 raft_consensus.cc:374] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.901908 32124 raft_consensus.cc:729] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14753c93125941acaad7283edc669052, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.902011 32124 consensus_queue.cc:260] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:29.902273 32124 ts_tablet_manager.cc:1398] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:29.902776 32122 tablet_bootstrap.cc:492] T 5702766b82f24e96a4cb5da5f051fc3f P ad5befacd63742a1b5b69fdacdb8cf8f: Bootstrap starting.
I20240502 14:00:29.903090 32123 tablet_bootstrap.cc:492] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc: No bootstrap required, opened a new log
I20240502 14:00:29.903157 32123 ts_tablet_manager.cc:1367] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 14:00:29.903566 32122 tablet_bootstrap.cc:654] T 5702766b82f24e96a4cb5da5f051fc3f P ad5befacd63742a1b5b69fdacdb8cf8f: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.904186 32127 tablet_bootstrap.cc:492] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929: Bootstrap starting.
I20240502 14:00:29.904556 32123 raft_consensus.cc:348] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:29.904644 32123 raft_consensus.cc:374] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.904683 32123 raft_consensus.cc:729] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.904811 32123 consensus_queue.cc:260] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:29.905103 32123 ts_tablet_manager.cc:1398] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:29.905261 32123 tablet_bootstrap.cc:492] T 8c77a03ce6a142debba75e0d387aa8ec P 8c61042437444f5f92c76109dfb229fc: Bootstrap starting.
I20240502 14:00:29.906144 32123 tablet_bootstrap.cc:654] T 8c77a03ce6a142debba75e0d387aa8ec P 8c61042437444f5f92c76109dfb229fc: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.906320 32127 tablet_bootstrap.cc:654] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.907449 32122 tablet_bootstrap.cc:492] T 5702766b82f24e96a4cb5da5f051fc3f P ad5befacd63742a1b5b69fdacdb8cf8f: No bootstrap required, opened a new log
I20240502 14:00:29.907531 32122 ts_tablet_manager.cc:1367] T 5702766b82f24e96a4cb5da5f051fc3f P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 14:00:29.907740 32123 tablet_bootstrap.cc:492] T 8c77a03ce6a142debba75e0d387aa8ec P 8c61042437444f5f92c76109dfb229fc: No bootstrap required, opened a new log
I20240502 14:00:29.907896 32123 ts_tablet_manager.cc:1367] T 8c77a03ce6a142debba75e0d387aa8ec P 8c61042437444f5f92c76109dfb229fc: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 14:00:29.907992 32122 raft_consensus.cc:348] T 5702766b82f24e96a4cb5da5f051fc3f P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:29.908087 32122 raft_consensus.cc:374] T 5702766b82f24e96a4cb5da5f051fc3f P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.908236 32122 raft_consensus.cc:729] T 5702766b82f24e96a4cb5da5f051fc3f P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.908356 32123 raft_consensus.cc:348] T 8c77a03ce6a142debba75e0d387aa8ec P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:29.908443 32123 raft_consensus.cc:374] T 8c77a03ce6a142debba75e0d387aa8ec P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.908398 32122 consensus_queue.cc:260] T 5702766b82f24e96a4cb5da5f051fc3f P ad5befacd63742a1b5b69fdacdb8cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:29.908474 32123 raft_consensus.cc:729] T 8c77a03ce6a142debba75e0d387aa8ec P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.908584 32123 consensus_queue.cc:260] T 8c77a03ce6a142debba75e0d387aa8ec P 8c61042437444f5f92c76109dfb229fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:29.908777 32127 tablet_bootstrap.cc:492] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929: No bootstrap required, opened a new log
I20240502 14:00:29.908768 32122 ts_tablet_manager.cc:1398] T 5702766b82f24e96a4cb5da5f051fc3f P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:29.908825 32123 ts_tablet_manager.cc:1398] T 8c77a03ce6a142debba75e0d387aa8ec P 8c61042437444f5f92c76109dfb229fc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:29.908847 32127 ts_tablet_manager.cc:1367] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:29.909236 32122 tablet_bootstrap.cc:492] T f571119d6f624fe88bf1effeaf466085 P ad5befacd63742a1b5b69fdacdb8cf8f: Bootstrap starting.
I20240502 14:00:29.909253 32127 raft_consensus.cc:348] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:29.909338 32127 raft_consensus.cc:374] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.909366 32127 raft_consensus.cc:729] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.909495 32127 consensus_queue.cc:260] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:29.909822 32127 ts_tablet_manager.cc:1398] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:29.910001 32127 tablet_bootstrap.cc:492] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929: Bootstrap starting.
I20240502 14:00:29.910049 32122 tablet_bootstrap.cc:654] T f571119d6f624fe88bf1effeaf466085 P ad5befacd63742a1b5b69fdacdb8cf8f: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.910816 32127 tablet_bootstrap.cc:654] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:29.910996 32122 tablet_bootstrap.cc:492] T f571119d6f624fe88bf1effeaf466085 P ad5befacd63742a1b5b69fdacdb8cf8f: No bootstrap required, opened a new log
I20240502 14:00:29.911067 32122 ts_tablet_manager.cc:1367] T f571119d6f624fe88bf1effeaf466085 P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:29.911442 32122 raft_consensus.cc:348] T f571119d6f624fe88bf1effeaf466085 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:29.911525 32122 raft_consensus.cc:374] T f571119d6f624fe88bf1effeaf466085 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.911557 32122 raft_consensus.cc:729] T f571119d6f624fe88bf1effeaf466085 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.911684 32122 consensus_queue.cc:260] T f571119d6f624fe88bf1effeaf466085 P ad5befacd63742a1b5b69fdacdb8cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:29.911854 32127 tablet_bootstrap.cc:492] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929: No bootstrap required, opened a new log
I20240502 14:00:29.911926 32127 ts_tablet_manager.cc:1367] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:29.911926 32122 ts_tablet_manager.cc:1398] T f571119d6f624fe88bf1effeaf466085 P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:29.912362 32127 raft_consensus.cc:348] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:29.912457 32127 raft_consensus.cc:374] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:29.912488 32127 raft_consensus.cc:729] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Initialized, Role: FOLLOWER
I20240502 14:00:29.912632 32127 consensus_queue.cc:260] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:29.912904 32127 ts_tablet_manager.cc:1398] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:30.008741 32120 raft_consensus.cc:482] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:30.008828 32120 raft_consensus.cc:504] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:30.009160 32120 leader_election.cc:288] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193), ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
I20240502 14:00:30.009418 31134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f571119d6f624fe88bf1effeaf466085" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc" is_pre_election: true
I20240502 14:00:30.009477 30865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f571119d6f624fe88bf1effeaf466085" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" is_pre_election: true
I20240502 14:00:30.009532 31134 raft_consensus.cc:2454] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 0.
I20240502 14:00:30.009603 30865 raft_consensus.cc:2454] T f571119d6f624fe88bf1effeaf466085 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 0.
I20240502 14:00:30.009824 31202 leader_election.cc:302] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 8c61042437444f5f92c76109dfb229fc; no voters:
I20240502 14:00:30.010015 32120 raft_consensus.cc:2781] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:30.010057 32120 raft_consensus.cc:482] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:30.010107 32120 raft_consensus.cc:3037] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.010883 32120 raft_consensus.cc:504] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:30.011154 32120 leader_election.cc:288] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 election: Requested vote from peers 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193), ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
I20240502 14:00:30.011416 30865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f571119d6f624fe88bf1effeaf466085" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
I20240502 14:00:30.011423 31134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f571119d6f624fe88bf1effeaf466085" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc"
I20240502 14:00:30.011521 31134 raft_consensus.cc:3037] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.011521 30865 raft_consensus.cc:3037] T f571119d6f624fe88bf1effeaf466085 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.012367 30865 raft_consensus.cc:2454] T f571119d6f624fe88bf1effeaf466085 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 1.
I20240502 14:00:30.012362 31134 raft_consensus.cc:2454] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 1.
I20240502 14:00:30.012632 31202 leader_election.cc:302] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 8c61042437444f5f92c76109dfb229fc; no voters:
I20240502 14:00:30.012786 32120 raft_consensus.cc:2781] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:30.012869 32120 raft_consensus.cc:686] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Becoming Leader. State: Replica: 14753c93125941acaad7283edc669052, State: Running, Role: LEADER
I20240502 14:00:30.012979 32120 consensus_queue.cc:237] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } }
I20240502 14:00:30.014425 30712 catalog_manager.cc:5484] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14753c93125941acaad7283edc669052 (127.24.66.68). New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:30.016487 31885 raft_consensus.cc:482] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:30.016564 31885 raft_consensus.cc:504] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:30.016822 31885 leader_election.cc:288] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14753c93125941acaad7283edc669052 (127.24.66.68:37029), 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:30.017083 31266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c77a03ce6a142debba75e0d387aa8ec" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052" is_pre_election: true
I20240502 14:00:30.017129 31134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c77a03ce6a142debba75e0d387aa8ec" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc" is_pre_election: true
I20240502 14:00:30.017189 31266 raft_consensus.cc:2454] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 0.
I20240502 14:00:30.017212 31134 raft_consensus.cc:2454] T 8c77a03ce6a142debba75e0d387aa8ec P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 0.
I20240502 14:00:30.017436 30936 leader_election.cc:302] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 1fab10dcfb8542fca2b965fed6c67929; no voters:
I20240502 14:00:30.017602 31885 raft_consensus.cc:2781] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:30.017663 31885 raft_consensus.cc:482] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:30.017692 31885 raft_consensus.cc:3037] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.017967 32136 raft_consensus.cc:482] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:30.018038 32136 raft_consensus.cc:504] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:30.018358 32136 leader_election.cc:288] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193), ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
I20240502 14:00:30.018370 31885 raft_consensus.cc:504] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:30.018709 30865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5702766b82f24e96a4cb5da5f051fc3f" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" is_pre_election: true
I20240502 14:00:30.018746 31885 leader_election.cc:288] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 election: Requested vote from peers 14753c93125941acaad7283edc669052 (127.24.66.68:37029), 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:30.018821 31134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5702766b82f24e96a4cb5da5f051fc3f" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc" is_pre_election: true
I20240502 14:00:30.018842 30865 raft_consensus.cc:2454] T 5702766b82f24e96a4cb5da5f051fc3f P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 0.
I20240502 14:00:30.018925 31134 raft_consensus.cc:2454] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 0.
I20240502 14:00:30.019109 30933 leader_election.cc:302] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fab10dcfb8542fca2b965fed6c67929, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:30.019162 31266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c77a03ce6a142debba75e0d387aa8ec" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052"
I20240502 14:00:30.019251 31266 raft_consensus.cc:3037] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.019254 31134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c77a03ce6a142debba75e0d387aa8ec" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc"
I20240502 14:00:30.019299 31885 raft_consensus.cc:2781] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:30.019340 31885 raft_consensus.cc:482] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:30.019347 31134 raft_consensus.cc:3037] T 8c77a03ce6a142debba75e0d387aa8ec P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.019384 31885 raft_consensus.cc:3037] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.019935 31266 raft_consensus.cc:2454] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 1.
I20240502 14:00:30.020146 31885 raft_consensus.cc:504] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:30.020187 31134 raft_consensus.cc:2454] T 8c77a03ce6a142debba75e0d387aa8ec P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 1.
I20240502 14:00:30.020267 30936 leader_election.cc:302] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 1fab10dcfb8542fca2b965fed6c67929; no voters:
I20240502 14:00:30.020447 32136 raft_consensus.cc:2781] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:30.020489 31885 leader_election.cc:288] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 election: Requested vote from peers 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193), ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
I20240502 14:00:30.020603 32136 raft_consensus.cc:686] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Becoming Leader. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Running, Role: LEADER
I20240502 14:00:30.020756 31134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5702766b82f24e96a4cb5da5f051fc3f" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc"
I20240502 14:00:30.020852 31134 raft_consensus.cc:3037] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.020849 30865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5702766b82f24e96a4cb5da5f051fc3f" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
I20240502 14:00:30.020938 30865 raft_consensus.cc:3037] T 5702766b82f24e96a4cb5da5f051fc3f P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.020972 32136 consensus_queue.cc:237] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:30.021592 31134 raft_consensus.cc:2454] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 1.
I20240502 14:00:30.021809 30865 raft_consensus.cc:2454] T 5702766b82f24e96a4cb5da5f051fc3f P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 1.
I20240502 14:00:30.021845 30936 leader_election.cc:302] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fab10dcfb8542fca2b965fed6c67929, 8c61042437444f5f92c76109dfb229fc; no voters:
I20240502 14:00:30.022302 32136 raft_consensus.cc:2781] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:30.022393 32136 raft_consensus.cc:686] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Becoming Leader. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Running, Role: LEADER
I20240502 14:00:30.022521 32136 consensus_queue.cc:237] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:30.022729 30707 catalog_manager.cc:5484] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66). New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:30.025241 30707 catalog_manager.cc:5484] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66). New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } }
W20240502 14:00:30.152993 32139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:30.153190 32139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:30.156069 32139 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20240502 14:00:30.225813 30865 raft_consensus.cc:1261] T f571119d6f624fe88bf1effeaf466085 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:30.226203 31134 raft_consensus.cc:1261] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:30.226943 31134 raft_consensus.cc:1261] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:30.227526 31885 consensus_queue.cc:1035] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:30.227555 31973 consensus_queue.cc:1035] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:30.227625 31134 raft_consensus.cc:1261] T 8c77a03ce6a142debba75e0d387aa8ec P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:30.228225 31266 raft_consensus.cc:1261] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:30.228274 32136 consensus_queue.cc:1035] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:30.230733 31973 consensus_queue.cc:1035] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:30.231127 30865 raft_consensus.cc:1261] T 5702766b82f24e96a4cb5da5f051fc3f P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:30.231482 31885 consensus_queue.cc:1035] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:30.237612 32136 consensus_queue.cc:1035] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 102.199 ms
time per row: 0.0510997 ms
I20240502 14:00:30.298997 30715 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38902:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:30.304176 30842 tablet_service.cc:1467] Processing CreateTablet for tablet efd8e3b804cf41bebf5e629b65571979 (DEFAULT_TABLE table=rebalance_test_table_4 [id=4cd0d0b0efb44a2c966dd894b6e48601]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.304448 31246 tablet_service.cc:1467] Processing CreateTablet for tablet 46b0a06884ad48da9b060d2810d5c782 (DEFAULT_TABLE table=rebalance_test_table_4 [id=4cd0d0b0efb44a2c966dd894b6e48601]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.304546 30842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efd8e3b804cf41bebf5e629b65571979. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.304693 31246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46b0a06884ad48da9b060d2810d5c782. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.304982 30981 tablet_service.cc:1467] Processing CreateTablet for tablet efd8e3b804cf41bebf5e629b65571979 (DEFAULT_TABLE table=rebalance_test_table_4 [id=4cd0d0b0efb44a2c966dd894b6e48601]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.305214 30981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efd8e3b804cf41bebf5e629b65571979. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.306876 31245 tablet_service.cc:1467] Processing CreateTablet for tablet 66e2378d5c0f400299a170df8a939577 (DEFAULT_TABLE table=rebalance_test_table_4 [id=4cd0d0b0efb44a2c966dd894b6e48601]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.307096 31245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66e2378d5c0f400299a170df8a939577. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.307210 30845 tablet_service.cc:1467] Processing CreateTablet for tablet 46b0a06884ad48da9b060d2810d5c782 (DEFAULT_TABLE table=rebalance_test_table_4 [id=4cd0d0b0efb44a2c966dd894b6e48601]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.307282 32127 tablet_bootstrap.cc:492] T efd8e3b804cf41bebf5e629b65571979 P 1fab10dcfb8542fca2b965fed6c67929: Bootstrap starting.
I20240502 14:00:30.307252 31115 tablet_service.cc:1467] Processing CreateTablet for tablet 66e2378d5c0f400299a170df8a939577 (DEFAULT_TABLE table=rebalance_test_table_4 [id=4cd0d0b0efb44a2c966dd894b6e48601]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.307422 30845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46b0a06884ad48da9b060d2810d5c782. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.307508 31115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66e2378d5c0f400299a170df8a939577. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.308087 32127 tablet_bootstrap.cc:654] T efd8e3b804cf41bebf5e629b65571979 P 1fab10dcfb8542fca2b965fed6c67929: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.308471 32124 tablet_bootstrap.cc:492] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052: Bootstrap starting.
I20240502 14:00:30.309033 32127 tablet_bootstrap.cc:492] T efd8e3b804cf41bebf5e629b65571979 P 1fab10dcfb8542fca2b965fed6c67929: No bootstrap required, opened a new log
I20240502 14:00:30.309105 32127 ts_tablet_manager.cc:1367] T efd8e3b804cf41bebf5e629b65571979 P 1fab10dcfb8542fca2b965fed6c67929: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:30.309264 30981 tablet_service.cc:1467] Processing CreateTablet for tablet 46b0a06884ad48da9b060d2810d5c782 (DEFAULT_TABLE table=rebalance_test_table_4 [id=4cd0d0b0efb44a2c966dd894b6e48601]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.309409 32124 tablet_bootstrap.cc:654] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.309479 30981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46b0a06884ad48da9b060d2810d5c782. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.309569 32127 raft_consensus.cc:348] T efd8e3b804cf41bebf5e629b65571979 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:30.309669 32127 raft_consensus.cc:374] T efd8e3b804cf41bebf5e629b65571979 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.309705 32127 raft_consensus.cc:729] T efd8e3b804cf41bebf5e629b65571979 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.309841 32127 consensus_queue.cc:260] T efd8e3b804cf41bebf5e629b65571979 P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:30.310019 32122 tablet_bootstrap.cc:492] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f: Bootstrap starting.
I20240502 14:00:30.310111 32127 ts_tablet_manager.cc:1398] T efd8e3b804cf41bebf5e629b65571979 P 1fab10dcfb8542fca2b965fed6c67929: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:30.310793 32122 tablet_bootstrap.cc:654] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.310995 32124 tablet_bootstrap.cc:492] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052: No bootstrap required, opened a new log
I20240502 14:00:30.311077 32124 ts_tablet_manager.cc:1367] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 14:00:30.311085 32127 tablet_bootstrap.cc:492] T 46b0a06884ad48da9b060d2810d5c782 P 1fab10dcfb8542fca2b965fed6c67929: Bootstrap starting.
I20240502 14:00:30.311553 31114 tablet_service.cc:1467] Processing CreateTablet for tablet efd8e3b804cf41bebf5e629b65571979 (DEFAULT_TABLE table=rebalance_test_table_4 [id=4cd0d0b0efb44a2c966dd894b6e48601]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.311689 32124 raft_consensus.cc:348] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:30.311789 32124 raft_consensus.cc:374] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.311775 32123 tablet_bootstrap.cc:492] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc: Bootstrap starting.
I20240502 14:00:30.311784 31114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efd8e3b804cf41bebf5e629b65571979. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.311828 32124 raft_consensus.cc:729] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14753c93125941acaad7283edc669052, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.311939 32127 tablet_bootstrap.cc:654] T 46b0a06884ad48da9b060d2810d5c782 P 1fab10dcfb8542fca2b965fed6c67929: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.311941 32124 consensus_queue.cc:260] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:30.312090 30843 tablet_service.cc:1467] Processing CreateTablet for tablet 66e2378d5c0f400299a170df8a939577 (DEFAULT_TABLE table=rebalance_test_table_4 [id=4cd0d0b0efb44a2c966dd894b6e48601]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.312188 32124 ts_tablet_manager.cc:1398] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:30.312310 30843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66e2378d5c0f400299a170df8a939577. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.313683 32124 tablet_bootstrap.cc:492] T 66e2378d5c0f400299a170df8a939577 P 14753c93125941acaad7283edc669052: Bootstrap starting.
I20240502 14:00:30.314321 32127 tablet_bootstrap.cc:492] T 46b0a06884ad48da9b060d2810d5c782 P 1fab10dcfb8542fca2b965fed6c67929: No bootstrap required, opened a new log
I20240502 14:00:30.314399 32127 ts_tablet_manager.cc:1367] T 46b0a06884ad48da9b060d2810d5c782 P 1fab10dcfb8542fca2b965fed6c67929: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:30.314591 32124 tablet_bootstrap.cc:654] T 66e2378d5c0f400299a170df8a939577 P 14753c93125941acaad7283edc669052: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.314846 32127 raft_consensus.cc:348] T 46b0a06884ad48da9b060d2810d5c782 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:30.314940 32127 raft_consensus.cc:374] T 46b0a06884ad48da9b060d2810d5c782 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.314970 32127 raft_consensus.cc:729] T 46b0a06884ad48da9b060d2810d5c782 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.315093 32127 consensus_queue.cc:260] T 46b0a06884ad48da9b060d2810d5c782 P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:30.315354 32123 tablet_bootstrap.cc:654] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.315366 32127 ts_tablet_manager.cc:1398] T 46b0a06884ad48da9b060d2810d5c782 P 1fab10dcfb8542fca2b965fed6c67929: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:30.316540 32123 tablet_bootstrap.cc:492] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc: No bootstrap required, opened a new log
I20240502 14:00:30.316623 32123 ts_tablet_manager.cc:1367] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20240502 14:00:30.317071 32123 raft_consensus.cc:348] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:30.317158 32123 raft_consensus.cc:374] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.317186 32123 raft_consensus.cc:729] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.317271 32124 tablet_bootstrap.cc:492] T 66e2378d5c0f400299a170df8a939577 P 14753c93125941acaad7283edc669052: No bootstrap required, opened a new log
I20240502 14:00:30.317312 32123 consensus_queue.cc:260] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:30.317363 32124 ts_tablet_manager.cc:1367] T 66e2378d5c0f400299a170df8a939577 P 14753c93125941acaad7283edc669052: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240502 14:00:30.317593 32123 ts_tablet_manager.cc:1398] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:30.317732 32123 tablet_bootstrap.cc:492] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc: Bootstrap starting.
I20240502 14:00:30.317760 32124 raft_consensus.cc:348] T 66e2378d5c0f400299a170df8a939577 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:30.317850 32124 raft_consensus.cc:374] T 66e2378d5c0f400299a170df8a939577 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.317881 32124 raft_consensus.cc:729] T 66e2378d5c0f400299a170df8a939577 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14753c93125941acaad7283edc669052, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.317992 32124 consensus_queue.cc:260] T 66e2378d5c0f400299a170df8a939577 P 14753c93125941acaad7283edc669052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:30.318253 32124 ts_tablet_manager.cc:1398] T 66e2378d5c0f400299a170df8a939577 P 14753c93125941acaad7283edc669052: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 14:00:30.318595 32123 tablet_bootstrap.cc:654] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.320019 32123 tablet_bootstrap.cc:492] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc: No bootstrap required, opened a new log
I20240502 14:00:30.320062 32122 tablet_bootstrap.cc:492] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f: No bootstrap required, opened a new log
I20240502 14:00:30.320151 32122 ts_tablet_manager.cc:1367] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20240502 14:00:30.320153 32123 ts_tablet_manager.cc:1367] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 14:00:30.320495 32122 raft_consensus.cc:348] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:30.320571 32122 raft_consensus.cc:374] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.320591 32122 raft_consensus.cc:729] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.320577 32123 raft_consensus.cc:348] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:30.320667 32123 raft_consensus.cc:374] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.320698 32123 raft_consensus.cc:729] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.320699 32122 consensus_queue.cc:260] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:30.320930 32122 ts_tablet_manager.cc:1398] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:30.321043 32122 tablet_bootstrap.cc:492] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f: Bootstrap starting.
I20240502 14:00:30.320809 32123 consensus_queue.cc:260] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:30.321635 32123 ts_tablet_manager.cc:1398] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:30.321851 32122 tablet_bootstrap.cc:654] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.322857 32122 tablet_bootstrap.cc:492] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f: No bootstrap required, opened a new log
I20240502 14:00:30.322938 32122 ts_tablet_manager.cc:1367] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:30.323341 32122 raft_consensus.cc:348] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:30.323429 32122 raft_consensus.cc:374] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.323459 32122 raft_consensus.cc:729] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.323570 32122 consensus_queue.cc:260] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:30.323848 32122 ts_tablet_manager.cc:1398] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:30.324033 32122 tablet_bootstrap.cc:492] T 46b0a06884ad48da9b060d2810d5c782 P ad5befacd63742a1b5b69fdacdb8cf8f: Bootstrap starting.
I20240502 14:00:30.325222 32122 tablet_bootstrap.cc:654] T 46b0a06884ad48da9b060d2810d5c782 P ad5befacd63742a1b5b69fdacdb8cf8f: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.326350 32122 tablet_bootstrap.cc:492] T 46b0a06884ad48da9b060d2810d5c782 P ad5befacd63742a1b5b69fdacdb8cf8f: No bootstrap required, opened a new log
I20240502 14:00:30.326438 32122 ts_tablet_manager.cc:1367] T 46b0a06884ad48da9b060d2810d5c782 P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:30.326774 32122 raft_consensus.cc:348] T 46b0a06884ad48da9b060d2810d5c782 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:30.326850 32122 raft_consensus.cc:374] T 46b0a06884ad48da9b060d2810d5c782 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.326870 32122 raft_consensus.cc:729] T 46b0a06884ad48da9b060d2810d5c782 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.326946 32122 consensus_queue.cc:260] T 46b0a06884ad48da9b060d2810d5c782 P ad5befacd63742a1b5b69fdacdb8cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:30.327239 32122 ts_tablet_manager.cc:1398] T 46b0a06884ad48da9b060d2810d5c782 P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:30.381616 31877 raft_consensus.cc:482] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:30.381722 31877 raft_consensus.cc:504] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:30.382032 31877 leader_election.cc:288] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553), 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:30.382405 31266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e2378d5c0f400299a170df8a939577" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052" is_pre_election: true
I20240502 14:00:30.382412 30864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e2378d5c0f400299a170df8a939577" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" is_pre_election: true
I20240502 14:00:30.382541 31266 raft_consensus.cc:2454] T 66e2378d5c0f400299a170df8a939577 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 0.
I20240502 14:00:30.382557 30864 raft_consensus.cc:2454] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 0.
I20240502 14:00:30.382828 31067 leader_election.cc:302] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 8c61042437444f5f92c76109dfb229fc, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:30.383030 31877 raft_consensus.cc:2781] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:30.383087 31877 raft_consensus.cc:482] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:30.383141 31877 raft_consensus.cc:3037] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.383936 31877 raft_consensus.cc:504] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:30.384238 31877 leader_election.cc:288] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 1 election: Requested vote from peers ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553), 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:30.384542 31266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e2378d5c0f400299a170df8a939577" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052"
I20240502 14:00:30.384550 30864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66e2378d5c0f400299a170df8a939577" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
I20240502 14:00:30.384635 31266 raft_consensus.cc:3037] T 66e2378d5c0f400299a170df8a939577 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.384646 30864 raft_consensus.cc:3037] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.385325 31266 raft_consensus.cc:2454] T 66e2378d5c0f400299a170df8a939577 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 1.
I20240502 14:00:30.385356 30864 raft_consensus.cc:2454] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 1.
I20240502 14:00:30.385597 31070 leader_election.cc:302] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 8c61042437444f5f92c76109dfb229fc; no voters:
I20240502 14:00:30.385751 31877 raft_consensus.cc:2781] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:30.385833 31877 raft_consensus.cc:686] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Becoming Leader. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Running, Role: LEADER
I20240502 14:00:30.385996 31877 consensus_queue.cc:237] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:30.387504 30715 catalog_manager.cc:5484] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c61042437444f5f92c76109dfb229fc (127.24.66.67). New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:30.393308 31880 raft_consensus.cc:482] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:30.393395 31880 raft_consensus.cc:504] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:30.393723 31880 leader_election.cc:288] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395), 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:30.394064 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efd8e3b804cf41bebf5e629b65571979" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929" is_pre_election: true
I20240502 14:00:30.394068 31133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efd8e3b804cf41bebf5e629b65571979" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc" is_pre_election: true
I20240502 14:00:30.394178 31133 raft_consensus.cc:2454] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 0.
I20240502 14:00:30.394188 31003 raft_consensus.cc:2454] T efd8e3b804cf41bebf5e629b65571979 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 0.
I20240502 14:00:30.394423 30798 leader_election.cc:302] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c61042437444f5f92c76109dfb229fc, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:30.394587 31880 raft_consensus.cc:2781] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:30.394650 31880 raft_consensus.cc:482] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:30.394692 31880 raft_consensus.cc:3037] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.395373 31880 raft_consensus.cc:504] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:30.395687 31880 leader_election.cc:288] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 election: Requested vote from peers 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395), 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:30.395967 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efd8e3b804cf41bebf5e629b65571979" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
I20240502 14:00:30.396014 31133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efd8e3b804cf41bebf5e629b65571979" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc"
I20240502 14:00:30.396098 31133 raft_consensus.cc:3037] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.396064 31003 raft_consensus.cc:3037] T efd8e3b804cf41bebf5e629b65571979 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.396907 31003 raft_consensus.cc:2454] T efd8e3b804cf41bebf5e629b65571979 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 1.
I20240502 14:00:30.396935 31133 raft_consensus.cc:2454] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 1.
I20240502 14:00:30.397162 30798 leader_election.cc:302] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c61042437444f5f92c76109dfb229fc, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:30.397315 31880 raft_consensus.cc:2781] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:30.397468 31880 raft_consensus.cc:686] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Becoming Leader. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Running, Role: LEADER
I20240502 14:00:30.397586 31880 consensus_queue.cc:237] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:30.398985 30715 catalog_manager.cc:5484] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: term changed from 0 to 1, leader changed from <none> to ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65). New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:30.566948 32120 raft_consensus.cc:482] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:30.567070 32120 raft_consensus.cc:504] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:30.567442 32120 leader_election.cc:288] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553), 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:30.567729 30864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46b0a06884ad48da9b060d2810d5c782" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" is_pre_election: true
I20240502 14:00:30.567744 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46b0a06884ad48da9b060d2810d5c782" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929" is_pre_election: true
I20240502 14:00:30.567864 30864 raft_consensus.cc:2454] T 46b0a06884ad48da9b060d2810d5c782 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 0.
I20240502 14:00:30.567870 31003 raft_consensus.cc:2454] T 46b0a06884ad48da9b060d2810d5c782 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 0.
I20240502 14:00:30.568110 31201 leader_election.cc:302] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 1fab10dcfb8542fca2b965fed6c67929; no voters:
I20240502 14:00:30.568275 32120 raft_consensus.cc:2781] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:30.568347 32120 raft_consensus.cc:482] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:30.568400 32120 raft_consensus.cc:3037] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.569188 32120 raft_consensus.cc:504] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:30.569499 32120 leader_election.cc:288] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 election: Requested vote from peers ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553), 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:30.569751 30864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46b0a06884ad48da9b060d2810d5c782" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
I20240502 14:00:30.569780 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46b0a06884ad48da9b060d2810d5c782" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
I20240502 14:00:30.569854 30864 raft_consensus.cc:3037] T 46b0a06884ad48da9b060d2810d5c782 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.569869 31003 raft_consensus.cc:3037] T 46b0a06884ad48da9b060d2810d5c782 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:30.570511 30864 raft_consensus.cc:2454] T 46b0a06884ad48da9b060d2810d5c782 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 1.
I20240502 14:00:30.570528 31003 raft_consensus.cc:2454] T 46b0a06884ad48da9b060d2810d5c782 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 1.
I20240502 14:00:30.570767 31201 leader_election.cc:302] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 1fab10dcfb8542fca2b965fed6c67929; no voters:
I20240502 14:00:30.570907 32120 raft_consensus.cc:2781] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:30.570992 32120 raft_consensus.cc:686] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Becoming Leader. State: Replica: 14753c93125941acaad7283edc669052, State: Running, Role: LEADER
I20240502 14:00:30.571142 32120 consensus_queue.cc:237] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:30.572616 30715 catalog_manager.cc:5484] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14753c93125941acaad7283edc669052 (127.24.66.68). New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:30.709026 32190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:30.709260 32190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:30.712386 32190 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20240502 14:00:30.787612 31133 raft_consensus.cc:1261] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:30.787676 31003 raft_consensus.cc:1261] T efd8e3b804cf41bebf5e629b65571979 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:30.787997 31880 consensus_queue.cc:1035] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:30.788264 31880 consensus_queue.cc:1035] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:30.791217 30864 raft_consensus.cc:1261] T 46b0a06884ad48da9b060d2810d5c782 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:30.791429 31002 raft_consensus.cc:1261] T 46b0a06884ad48da9b060d2810d5c782 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:30.791545 32120 consensus_queue.cc:1035] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:30.792420 32120 consensus_queue.cc:1035] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:30.796851 31266 raft_consensus.cc:1261] T 66e2378d5c0f400299a170df8a939577 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:30.797523 31877 consensus_queue.cc:1035] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:30.801059 30865 raft_consensus.cc:1261] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:30.802033 31877 consensus_queue.cc:1035] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:30.832355 32226 mvcc.cc:205] Tried to move back new op lower bound from 7023240932514955264 to 7023240930862178304. Current Snapshot: MvccSnapshot[applied={T|T < 7023240932514955264}]
INSERT report
rows total: 2000
time total: 106.742 ms
time per row: 0.0533712 ms
I20240502 14:00:30.858884 30715 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38902:
name: "rebalance_test_table_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:30.863740 30981 tablet_service.cc:1467] Processing CreateTablet for tablet cc64ea7db2064251bed364b800b36b87 (DEFAULT_TABLE table=rebalance_test_table_5 [id=b000c5e9f8984019bef07f52b5bff49d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.863854 31114 tablet_service.cc:1467] Processing CreateTablet for tablet cc64ea7db2064251bed364b800b36b87 (DEFAULT_TABLE table=rebalance_test_table_5 [id=b000c5e9f8984019bef07f52b5bff49d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.864035 30981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc64ea7db2064251bed364b800b36b87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.864115 31114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc64ea7db2064251bed364b800b36b87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.864516 30845 tablet_service.cc:1467] Processing CreateTablet for tablet e109ef8f5d764df7969642dc6de361a2 (DEFAULT_TABLE table=rebalance_test_table_5 [id=b000c5e9f8984019bef07f52b5bff49d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.864778 30845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e109ef8f5d764df7969642dc6de361a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.865010 31113 tablet_service.cc:1467] Processing CreateTablet for tablet e109ef8f5d764df7969642dc6de361a2 (DEFAULT_TABLE table=rebalance_test_table_5 [id=b000c5e9f8984019bef07f52b5bff49d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.865235 31113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e109ef8f5d764df7969642dc6de361a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.866565 31115 tablet_service.cc:1467] Processing CreateTablet for tablet 39239edeafb84b5caaa5dceea06fd4b0 (DEFAULT_TABLE table=rebalance_test_table_5 [id=b000c5e9f8984019bef07f52b5bff49d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.866765 31115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39239edeafb84b5caaa5dceea06fd4b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.867439 32255 tablet_bootstrap.cc:492] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc: Bootstrap starting.
I20240502 14:00:30.867712 31245 tablet_service.cc:1467] Processing CreateTablet for tablet cc64ea7db2064251bed364b800b36b87 (DEFAULT_TABLE table=rebalance_test_table_5 [id=b000c5e9f8984019bef07f52b5bff49d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.867707 31246 tablet_service.cc:1467] Processing CreateTablet for tablet 39239edeafb84b5caaa5dceea06fd4b0 (DEFAULT_TABLE table=rebalance_test_table_5 [id=b000c5e9f8984019bef07f52b5bff49d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.867978 31245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc64ea7db2064251bed364b800b36b87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.868104 31246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39239edeafb84b5caaa5dceea06fd4b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.868234 32255 tablet_bootstrap.cc:654] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.869199 32255 tablet_bootstrap.cc:492] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc: No bootstrap required, opened a new log
I20240502 14:00:30.869280 32255 ts_tablet_manager.cc:1367] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:30.869670 32255 raft_consensus.cc:348] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:30.869771 32255 raft_consensus.cc:374] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.869753 30982 tablet_service.cc:1467] Processing CreateTablet for tablet e109ef8f5d764df7969642dc6de361a2 (DEFAULT_TABLE table=rebalance_test_table_5 [id=b000c5e9f8984019bef07f52b5bff49d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.869807 32255 raft_consensus.cc:729] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.869946 30982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e109ef8f5d764df7969642dc6de361a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.869926 32255 consensus_queue.cc:260] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:30.870199 32255 ts_tablet_manager.cc:1398] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:30.870352 32255 tablet_bootstrap.cc:492] T e109ef8f5d764df7969642dc6de361a2 P 8c61042437444f5f92c76109dfb229fc: Bootstrap starting.
I20240502 14:00:30.870393 32257 tablet_bootstrap.cc:492] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052: Bootstrap starting.
I20240502 14:00:30.871066 32254 tablet_bootstrap.cc:492] T e109ef8f5d764df7969642dc6de361a2 P ad5befacd63742a1b5b69fdacdb8cf8f: Bootstrap starting.
I20240502 14:00:30.871127 32257 tablet_bootstrap.cc:654] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.871124 32255 tablet_bootstrap.cc:654] T e109ef8f5d764df7969642dc6de361a2 P 8c61042437444f5f92c76109dfb229fc: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.872071 32254 tablet_bootstrap.cc:654] T e109ef8f5d764df7969642dc6de361a2 P ad5befacd63742a1b5b69fdacdb8cf8f: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.872189 32257 tablet_bootstrap.cc:492] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052: No bootstrap required, opened a new log
I20240502 14:00:30.872275 32257 ts_tablet_manager.cc:1367] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:30.872282 32255 tablet_bootstrap.cc:492] T e109ef8f5d764df7969642dc6de361a2 P 8c61042437444f5f92c76109dfb229fc: No bootstrap required, opened a new log
I20240502 14:00:30.872372 32255 ts_tablet_manager.cc:1367] T e109ef8f5d764df7969642dc6de361a2 P 8c61042437444f5f92c76109dfb229fc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:30.872725 32255 raft_consensus.cc:348] T e109ef8f5d764df7969642dc6de361a2 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:30.872725 32257 raft_consensus.cc:348] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:30.872826 32257 raft_consensus.cc:374] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.872826 32255 raft_consensus.cc:374] T e109ef8f5d764df7969642dc6de361a2 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.872865 32257 raft_consensus.cc:729] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14753c93125941acaad7283edc669052, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.872865 32255 raft_consensus.cc:729] T e109ef8f5d764df7969642dc6de361a2 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.872974 32255 consensus_queue.cc:260] T e109ef8f5d764df7969642dc6de361a2 P 8c61042437444f5f92c76109dfb229fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:30.872974 32257 consensus_queue.cc:260] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:30.873217 32257 ts_tablet_manager.cc:1398] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:30.873217 32255 ts_tablet_manager.cc:1398] T e109ef8f5d764df7969642dc6de361a2 P 8c61042437444f5f92c76109dfb229fc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:30.873342 32257 tablet_bootstrap.cc:492] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052: Bootstrap starting.
I20240502 14:00:30.873340 32255 tablet_bootstrap.cc:492] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc: Bootstrap starting.
I20240502 14:00:30.873353 30981 tablet_service.cc:1467] Processing CreateTablet for tablet 39239edeafb84b5caaa5dceea06fd4b0 (DEFAULT_TABLE table=rebalance_test_table_5 [id=b000c5e9f8984019bef07f52b5bff49d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:30.873584 30981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39239edeafb84b5caaa5dceea06fd4b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:30.874114 32257 tablet_bootstrap.cc:654] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.874114 32255 tablet_bootstrap.cc:654] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.875605 32254 tablet_bootstrap.cc:492] T e109ef8f5d764df7969642dc6de361a2 P ad5befacd63742a1b5b69fdacdb8cf8f: No bootstrap required, opened a new log
I20240502 14:00:30.875690 32254 ts_tablet_manager.cc:1367] T e109ef8f5d764df7969642dc6de361a2 P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 14:00:30.876130 32254 raft_consensus.cc:348] T e109ef8f5d764df7969642dc6de361a2 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:30.876221 32254 raft_consensus.cc:374] T e109ef8f5d764df7969642dc6de361a2 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.876256 32254 raft_consensus.cc:729] T e109ef8f5d764df7969642dc6de361a2 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.876319 32255 tablet_bootstrap.cc:492] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc: No bootstrap required, opened a new log
I20240502 14:00:30.876428 32255 ts_tablet_manager.cc:1367] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:30.876399 32254 consensus_queue.cc:260] T e109ef8f5d764df7969642dc6de361a2 P ad5befacd63742a1b5b69fdacdb8cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:30.876677 32254 ts_tablet_manager.cc:1398] T e109ef8f5d764df7969642dc6de361a2 P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:30.876803 32255 raft_consensus.cc:348] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:30.876902 32255 raft_consensus.cc:374] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.876942 32255 raft_consensus.cc:729] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.877007 32253 tablet_bootstrap.cc:492] T cc64ea7db2064251bed364b800b36b87 P 1fab10dcfb8542fca2b965fed6c67929: Bootstrap starting.
I20240502 14:00:30.877079 32255 consensus_queue.cc:260] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:30.877612 32255 ts_tablet_manager.cc:1398] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:30.877864 32253 tablet_bootstrap.cc:654] T cc64ea7db2064251bed364b800b36b87 P 1fab10dcfb8542fca2b965fed6c67929: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.878794 32257 tablet_bootstrap.cc:492] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052: No bootstrap required, opened a new log
I20240502 14:00:30.878870 32257 ts_tablet_manager.cc:1367] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 14:00:30.878876 32253 tablet_bootstrap.cc:492] T cc64ea7db2064251bed364b800b36b87 P 1fab10dcfb8542fca2b965fed6c67929: No bootstrap required, opened a new log
I20240502 14:00:30.878947 32253 ts_tablet_manager.cc:1367] T cc64ea7db2064251bed364b800b36b87 P 1fab10dcfb8542fca2b965fed6c67929: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:30.879209 32257 raft_consensus.cc:348] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:30.879325 32257 raft_consensus.cc:374] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.879361 32257 raft_consensus.cc:729] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14753c93125941acaad7283edc669052, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.879352 32253 raft_consensus.cc:348] T cc64ea7db2064251bed364b800b36b87 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:30.879468 32253 raft_consensus.cc:374] T cc64ea7db2064251bed364b800b36b87 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.879513 32253 raft_consensus.cc:729] T cc64ea7db2064251bed364b800b36b87 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.879493 32257 consensus_queue.cc:260] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:30.879747 32257 ts_tablet_manager.cc:1398] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:30.879730 32253 consensus_queue.cc:260] T cc64ea7db2064251bed364b800b36b87 P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:30.879988 32253 ts_tablet_manager.cc:1398] T cc64ea7db2064251bed364b800b36b87 P 1fab10dcfb8542fca2b965fed6c67929: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:30.880676 32253 tablet_bootstrap.cc:492] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929: Bootstrap starting.
I20240502 14:00:30.881455 32253 tablet_bootstrap.cc:654] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.882304 32253 tablet_bootstrap.cc:492] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929: No bootstrap required, opened a new log
I20240502 14:00:30.882378 32253 ts_tablet_manager.cc:1367] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:30.882730 32253 raft_consensus.cc:348] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:30.882833 32253 raft_consensus.cc:374] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.882876 32253 raft_consensus.cc:729] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.882973 32253 consensus_queue.cc:260] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:30.883179 32253 ts_tablet_manager.cc:1398] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:30.883342 32253 tablet_bootstrap.cc:492] T 39239edeafb84b5caaa5dceea06fd4b0 P 1fab10dcfb8542fca2b965fed6c67929: Bootstrap starting.
I20240502 14:00:30.884193 32253 tablet_bootstrap.cc:654] T 39239edeafb84b5caaa5dceea06fd4b0 P 1fab10dcfb8542fca2b965fed6c67929: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:30.885255 32253 tablet_bootstrap.cc:492] T 39239edeafb84b5caaa5dceea06fd4b0 P 1fab10dcfb8542fca2b965fed6c67929: No bootstrap required, opened a new log
I20240502 14:00:30.885355 32253 ts_tablet_manager.cc:1367] T 39239edeafb84b5caaa5dceea06fd4b0 P 1fab10dcfb8542fca2b965fed6c67929: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:30.885792 32253 raft_consensus.cc:348] T 39239edeafb84b5caaa5dceea06fd4b0 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:30.885882 32253 raft_consensus.cc:374] T 39239edeafb84b5caaa5dceea06fd4b0 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:30.885903 32253 raft_consensus.cc:729] T 39239edeafb84b5caaa5dceea06fd4b0 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Initialized, Role: FOLLOWER
I20240502 14:00:30.885991 32253 consensus_queue.cc:260] T 39239edeafb84b5caaa5dceea06fd4b0 P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:30.886178 32253 ts_tablet_manager.cc:1398] T 39239edeafb84b5caaa5dceea06fd4b0 P 1fab10dcfb8542fca2b965fed6c67929: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:31.034745 32230 raft_consensus.cc:482] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:31.034842 32230 raft_consensus.cc:504] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:31.035257 32230 leader_election.cc:288] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553), 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:31.035516 30864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e109ef8f5d764df7969642dc6de361a2" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" is_pre_election: true
I20240502 14:00:31.035660 30864 raft_consensus.cc:2454] T e109ef8f5d764df7969642dc6de361a2 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 0.
I20240502 14:00:31.035683 31133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e109ef8f5d764df7969642dc6de361a2" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc" is_pre_election: true
I20240502 14:00:31.035809 31133 raft_consensus.cc:2454] T e109ef8f5d764df7969642dc6de361a2 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 0.
I20240502 14:00:31.035943 30933 leader_election.cc:302] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fab10dcfb8542fca2b965fed6c67929, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:31.036090 32230 raft_consensus.cc:2781] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:31.036145 32230 raft_consensus.cc:482] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:31.036191 32230 raft_consensus.cc:3037] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.037046 32230 raft_consensus.cc:504] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:31.037357 32230 leader_election.cc:288] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 election: Requested vote from peers ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553), 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:31.037613 30864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e109ef8f5d764df7969642dc6de361a2" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
I20240502 14:00:31.037673 31133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e109ef8f5d764df7969642dc6de361a2" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc"
I20240502 14:00:31.037765 30864 raft_consensus.cc:3037] T e109ef8f5d764df7969642dc6de361a2 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.037786 31133 raft_consensus.cc:3037] T e109ef8f5d764df7969642dc6de361a2 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.038545 31133 raft_consensus.cc:2454] T e109ef8f5d764df7969642dc6de361a2 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 1.
I20240502 14:00:31.038612 30864 raft_consensus.cc:2454] T e109ef8f5d764df7969642dc6de361a2 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 1.
I20240502 14:00:31.038784 30936 leader_election.cc:302] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fab10dcfb8542fca2b965fed6c67929, 8c61042437444f5f92c76109dfb229fc; no voters:
I20240502 14:00:31.038935 32230 raft_consensus.cc:2781] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:31.039034 32230 raft_consensus.cc:686] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Becoming Leader. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Running, Role: LEADER
I20240502 14:00:31.039162 32230 consensus_queue.cc:237] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:31.040583 30715 catalog_manager.cc:5484] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66). New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:31.059942 32016 raft_consensus.cc:482] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:31.060025 32016 raft_consensus.cc:504] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:31.060350 32016 leader_election.cc:288] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395), 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:31.060631 31133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39239edeafb84b5caaa5dceea06fd4b0" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc" is_pre_election: true
I20240502 14:00:31.060649 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39239edeafb84b5caaa5dceea06fd4b0" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929" is_pre_election: true
I20240502 14:00:31.060740 31133 raft_consensus.cc:2454] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 0.
I20240502 14:00:31.060743 31003 raft_consensus.cc:2454] T 39239edeafb84b5caaa5dceea06fd4b0 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 0.
I20240502 14:00:31.060974 31202 leader_election.cc:302] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 8c61042437444f5f92c76109dfb229fc; no voters:
I20240502 14:00:31.061139 32016 raft_consensus.cc:2781] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:31.061193 32016 raft_consensus.cc:482] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:31.061218 32016 raft_consensus.cc:3037] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.061928 32016 raft_consensus.cc:504] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:31.062202 32016 leader_election.cc:288] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 election: Requested vote from peers 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395), 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:31.062507 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39239edeafb84b5caaa5dceea06fd4b0" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
I20240502 14:00:31.062508 31133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39239edeafb84b5caaa5dceea06fd4b0" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc"
I20240502 14:00:31.062606 31003 raft_consensus.cc:3037] T 39239edeafb84b5caaa5dceea06fd4b0 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.062606 31133 raft_consensus.cc:3037] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.063380 31133 raft_consensus.cc:2454] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 1.
I20240502 14:00:31.063380 31003 raft_consensus.cc:2454] T 39239edeafb84b5caaa5dceea06fd4b0 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 1.
I20240502 14:00:31.063629 31202 leader_election.cc:302] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 8c61042437444f5f92c76109dfb229fc; no voters:
I20240502 14:00:31.063783 32016 raft_consensus.cc:2781] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:31.063860 32016 raft_consensus.cc:686] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Becoming Leader. State: Replica: 14753c93125941acaad7283edc669052, State: Running, Role: LEADER
I20240502 14:00:31.063982 32016 consensus_queue.cc:237] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:31.065390 30715 catalog_manager.cc:5484] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14753c93125941acaad7283edc669052 (127.24.66.68). New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:31.077122 32250 raft_consensus.cc:482] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:31.077207 32250 raft_consensus.cc:504] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:31.077622 32250 leader_election.cc:288] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395), 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:31.077960 31266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc64ea7db2064251bed364b800b36b87" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052" is_pre_election: true
I20240502 14:00:31.078085 31266 raft_consensus.cc:2454] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 0.
I20240502 14:00:31.078359 31070 leader_election.cc:302] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 8c61042437444f5f92c76109dfb229fc; no voters:
I20240502 14:00:31.078516 32250 raft_consensus.cc:2781] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:31.078567 32250 raft_consensus.cc:482] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:31.078622 32250 raft_consensus.cc:3037] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.079264 32250 raft_consensus.cc:504] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:31.079563 32250 leader_election.cc:288] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 1 election: Requested vote from peers 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395), 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:31.079864 31266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc64ea7db2064251bed364b800b36b87" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052"
I20240502 14:00:31.079963 31266 raft_consensus.cc:3037] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.080829 31266 raft_consensus.cc:2454] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 1.
I20240502 14:00:31.081079 31070 leader_election.cc:302] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 8c61042437444f5f92c76109dfb229fc; no voters:
I20240502 14:00:31.081115 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc64ea7db2064251bed364b800b36b87" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929" is_pre_election: true
I20240502 14:00:31.081151 31002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc64ea7db2064251bed364b800b36b87" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
I20240502 14:00:31.081223 31002 raft_consensus.cc:3037] T cc64ea7db2064251bed364b800b36b87 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.081430 32250 raft_consensus.cc:2781] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:31.081514 32250 raft_consensus.cc:686] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Becoming Leader. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Running, Role: LEADER
I20240502 14:00:31.081630 32250 consensus_queue.cc:237] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:31.083025 30715 catalog_manager.cc:5484] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c61042437444f5f92c76109dfb229fc (127.24.66.67). New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:31.101047 31002 raft_consensus.cc:2454] T cc64ea7db2064251bed364b800b36b87 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 1.
W20240502 14:00:31.206148 32272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:31.206373 32272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:31.209225 32272 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_5'
I20240502 14:00:31.287228 31266 raft_consensus.cc:1261] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:31.287662 31877 consensus_queue.cc:1035] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:31.288162 31002 raft_consensus.cc:1261] T cc64ea7db2064251bed364b800b36b87 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:31.289134 31133 raft_consensus.cc:1261] T e109ef8f5d764df7969642dc6de361a2 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:31.293684 30864 raft_consensus.cc:1261] T e109ef8f5d764df7969642dc6de361a2 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:31.293676 32230 consensus_queue.cc:1035] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:31.294231 31877 consensus_queue.cc:1035] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:31.296294 32174 consensus_queue.cc:1035] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:31.297792 31134 raft_consensus.cc:1261] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:31.298156 32120 consensus_queue.cc:1035] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:31.301538 31003 raft_consensus.cc:1261] T 39239edeafb84b5caaa5dceea06fd4b0 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:31.305819 32166 mvcc.cc:205] Tried to move back new op lower bound from 7023240934548025344 to 7023240933711450112. Current Snapshot: MvccSnapshot[applied={T|T < 7023240934548025344}]
I20240502 14:00:31.307479 32016 consensus_queue.cc:1035] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:31.307982 32236 mvcc.cc:205] Tried to move back new op lower bound from 7023240934548025344 to 7023240933711450112. Current Snapshot: MvccSnapshot[applied={T|T < 7023240934548025344}]
I20240502 14:00:31.325667 32225 mvcc.cc:205] Tried to move back new op lower bound from 7023240934576807936 to 7023240933537566720. Current Snapshot: MvccSnapshot[applied={T|T < 7023240934553882624}]
INSERT report
rows total: 2000
time total: 111.52 ms
time per row: 0.0557602 ms
I20240502 14:00:31.359943 30707 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38902:
name: "rebalance_test_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:31.364982 30845 tablet_service.cc:1467] Processing CreateTablet for tablet b5b3cf5fbe3d4d999cbf910c8e0ca62e (DEFAULT_TABLE table=rebalance_test_table_6 [id=99c461286f6a4e4f81cc54ae6fc11b21]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.365041 31245 tablet_service.cc:1467] Processing CreateTablet for tablet b5b3cf5fbe3d4d999cbf910c8e0ca62e (DEFAULT_TABLE table=rebalance_test_table_6 [id=99c461286f6a4e4f81cc54ae6fc11b21]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.365272 30845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5b3cf5fbe3d4d999cbf910c8e0ca62e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.365272 31245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5b3cf5fbe3d4d999cbf910c8e0ca62e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.365259 31246 tablet_service.cc:1467] Processing CreateTablet for tablet da2f3b3481da476d98c97bf35f055c2e (DEFAULT_TABLE table=rebalance_test_table_6 [id=99c461286f6a4e4f81cc54ae6fc11b21]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.365725 31246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da2f3b3481da476d98c97bf35f055c2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.366106 30843 tablet_service.cc:1467] Processing CreateTablet for tablet da2f3b3481da476d98c97bf35f055c2e (DEFAULT_TABLE table=rebalance_test_table_6 [id=99c461286f6a4e4f81cc54ae6fc11b21]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.366130 31244 tablet_service.cc:1467] Processing CreateTablet for tablet 954fa01ea4d04842b0218b33de347aa1 (DEFAULT_TABLE table=rebalance_test_table_6 [id=99c461286f6a4e4f81cc54ae6fc11b21]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.366303 30843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da2f3b3481da476d98c97bf35f055c2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.366355 31244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 954fa01ea4d04842b0218b33de347aa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.366739 30842 tablet_service.cc:1467] Processing CreateTablet for tablet 954fa01ea4d04842b0218b33de347aa1 (DEFAULT_TABLE table=rebalance_test_table_6 [id=99c461286f6a4e4f81cc54ae6fc11b21]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.366935 30842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 954fa01ea4d04842b0218b33de347aa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.367398 32257 tablet_bootstrap.cc:492] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 14753c93125941acaad7283edc669052: Bootstrap starting.
I20240502 14:00:31.368121 32257 tablet_bootstrap.cc:654] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 14753c93125941acaad7283edc669052: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.368839 30981 tablet_service.cc:1467] Processing CreateTablet for tablet b5b3cf5fbe3d4d999cbf910c8e0ca62e (DEFAULT_TABLE table=rebalance_test_table_6 [id=99c461286f6a4e4f81cc54ae6fc11b21]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.369093 30981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5b3cf5fbe3d4d999cbf910c8e0ca62e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.369091 31115 tablet_service.cc:1467] Processing CreateTablet for tablet da2f3b3481da476d98c97bf35f055c2e (DEFAULT_TABLE table=rebalance_test_table_6 [id=99c461286f6a4e4f81cc54ae6fc11b21]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.369341 31115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da2f3b3481da476d98c97bf35f055c2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.369437 30982 tablet_service.cc:1467] Processing CreateTablet for tablet 954fa01ea4d04842b0218b33de347aa1 (DEFAULT_TABLE table=rebalance_test_table_6 [id=99c461286f6a4e4f81cc54ae6fc11b21]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:31.369652 30982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 954fa01ea4d04842b0218b33de347aa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:31.371127 32253 tablet_bootstrap.cc:492] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 1fab10dcfb8542fca2b965fed6c67929: Bootstrap starting.
I20240502 14:00:31.371397 32255 tablet_bootstrap.cc:492] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc: Bootstrap starting.
I20240502 14:00:31.371955 32253 tablet_bootstrap.cc:654] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 1fab10dcfb8542fca2b965fed6c67929: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.372212 32257 tablet_bootstrap.cc:492] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 14753c93125941acaad7283edc669052: No bootstrap required, opened a new log
I20240502 14:00:31.372238 32255 tablet_bootstrap.cc:654] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.372270 32257 ts_tablet_manager.cc:1367] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 14753c93125941acaad7283edc669052: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:31.372625 32257 raft_consensus.cc:348] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:31.372718 32257 raft_consensus.cc:374] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:31.372752 32257 raft_consensus.cc:729] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14753c93125941acaad7283edc669052, State: Initialized, Role: FOLLOWER
I20240502 14:00:31.372874 32257 consensus_queue.cc:260] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 14753c93125941acaad7283edc669052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:31.373150 32257 ts_tablet_manager.cc:1398] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 14753c93125941acaad7283edc669052: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:31.373307 32257 tablet_bootstrap.cc:492] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052: Bootstrap starting.
I20240502 14:00:31.373592 32255 tablet_bootstrap.cc:492] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc: No bootstrap required, opened a new log
I20240502 14:00:31.373670 32255 ts_tablet_manager.cc:1367] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:31.374099 32253 tablet_bootstrap.cc:492] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 1fab10dcfb8542fca2b965fed6c67929: No bootstrap required, opened a new log
I20240502 14:00:31.374122 32257 tablet_bootstrap.cc:654] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.374058 32255 raft_consensus.cc:348] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:31.374152 32253 ts_tablet_manager.cc:1367] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 1fab10dcfb8542fca2b965fed6c67929: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 14:00:31.374155 32255 raft_consensus.cc:374] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:31.374186 32255 raft_consensus.cc:729] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Initialized, Role: FOLLOWER
I20240502 14:00:31.374317 32255 consensus_queue.cc:260] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:31.374532 32253 raft_consensus.cc:348] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:31.374621 32253 raft_consensus.cc:374] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:31.374632 32255 ts_tablet_manager.cc:1398] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:31.374679 32253 raft_consensus.cc:729] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Initialized, Role: FOLLOWER
I20240502 14:00:31.374807 32253 consensus_queue.cc:260] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:31.375082 32253 ts_tablet_manager.cc:1398] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 1fab10dcfb8542fca2b965fed6c67929: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:31.375342 32253 tablet_bootstrap.cc:492] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929: Bootstrap starting.
I20240502 14:00:31.375917 32257 tablet_bootstrap.cc:492] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052: No bootstrap required, opened a new log
I20240502 14:00:31.375995 32257 ts_tablet_manager.cc:1367] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:31.376237 32253 tablet_bootstrap.cc:654] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.376488 32257 raft_consensus.cc:348] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:31.376590 32257 raft_consensus.cc:374] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:31.376622 32257 raft_consensus.cc:729] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14753c93125941acaad7283edc669052, State: Initialized, Role: FOLLOWER
I20240502 14:00:31.376768 32257 consensus_queue.cc:260] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:31.377064 32257 ts_tablet_manager.cc:1398] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:31.377234 32257 tablet_bootstrap.cc:492] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052: Bootstrap starting.
I20240502 14:00:31.377665 32253 tablet_bootstrap.cc:492] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929: No bootstrap required, opened a new log
I20240502 14:00:31.377764 32253 ts_tablet_manager.cc:1367] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:31.378168 32253 raft_consensus.cc:348] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:31.378260 32257 tablet_bootstrap.cc:654] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.378278 32253 raft_consensus.cc:374] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:31.378309 32253 raft_consensus.cc:729] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Initialized, Role: FOLLOWER
I20240502 14:00:31.378444 32253 consensus_queue.cc:260] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:31.378737 32253 ts_tablet_manager.cc:1398] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:31.379173 32257 tablet_bootstrap.cc:492] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052: No bootstrap required, opened a new log
I20240502 14:00:31.379245 32257 ts_tablet_manager.cc:1367] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:31.379688 32257 raft_consensus.cc:348] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:31.379779 32257 raft_consensus.cc:374] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:31.379812 32257 raft_consensus.cc:729] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14753c93125941acaad7283edc669052, State: Initialized, Role: FOLLOWER
I20240502 14:00:31.379925 32257 consensus_queue.cc:260] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:31.380149 32257 ts_tablet_manager.cc:1398] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:31.382474 32316 tablet_bootstrap.cc:492] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f: Bootstrap starting.
I20240502 14:00:31.383387 32316 tablet_bootstrap.cc:654] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.384619 32316 tablet_bootstrap.cc:492] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f: No bootstrap required, opened a new log
I20240502 14:00:31.384702 32316 ts_tablet_manager.cc:1367] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:31.385062 32316 raft_consensus.cc:348] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:31.385143 32316 raft_consensus.cc:374] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:31.385178 32316 raft_consensus.cc:729] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Initialized, Role: FOLLOWER
I20240502 14:00:31.385298 32316 consensus_queue.cc:260] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:31.385511 32316 ts_tablet_manager.cc:1398] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:31.385663 32316 tablet_bootstrap.cc:492] T 954fa01ea4d04842b0218b33de347aa1 P ad5befacd63742a1b5b69fdacdb8cf8f: Bootstrap starting.
I20240502 14:00:31.386472 32316 tablet_bootstrap.cc:654] T 954fa01ea4d04842b0218b33de347aa1 P ad5befacd63742a1b5b69fdacdb8cf8f: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.387475 32316 tablet_bootstrap.cc:492] T 954fa01ea4d04842b0218b33de347aa1 P ad5befacd63742a1b5b69fdacdb8cf8f: No bootstrap required, opened a new log
I20240502 14:00:31.387547 32316 ts_tablet_manager.cc:1367] T 954fa01ea4d04842b0218b33de347aa1 P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:31.387959 32316 raft_consensus.cc:348] T 954fa01ea4d04842b0218b33de347aa1 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:31.388067 32316 raft_consensus.cc:374] T 954fa01ea4d04842b0218b33de347aa1 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:31.388106 32316 raft_consensus.cc:729] T 954fa01ea4d04842b0218b33de347aa1 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Initialized, Role: FOLLOWER
I20240502 14:00:31.388229 32316 consensus_queue.cc:260] T 954fa01ea4d04842b0218b33de347aa1 P ad5befacd63742a1b5b69fdacdb8cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:31.388509 32316 ts_tablet_manager.cc:1398] T 954fa01ea4d04842b0218b33de347aa1 P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:31.388736 32316 tablet_bootstrap.cc:492] T da2f3b3481da476d98c97bf35f055c2e P ad5befacd63742a1b5b69fdacdb8cf8f: Bootstrap starting.
I20240502 14:00:31.389400 32316 tablet_bootstrap.cc:654] T da2f3b3481da476d98c97bf35f055c2e P ad5befacd63742a1b5b69fdacdb8cf8f: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:31.390503 32316 tablet_bootstrap.cc:492] T da2f3b3481da476d98c97bf35f055c2e P ad5befacd63742a1b5b69fdacdb8cf8f: No bootstrap required, opened a new log
I20240502 14:00:31.390604 32316 ts_tablet_manager.cc:1367] T da2f3b3481da476d98c97bf35f055c2e P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:31.391010 32316 raft_consensus.cc:348] T da2f3b3481da476d98c97bf35f055c2e P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:31.391106 32316 raft_consensus.cc:374] T da2f3b3481da476d98c97bf35f055c2e P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:31.391139 32316 raft_consensus.cc:729] T da2f3b3481da476d98c97bf35f055c2e P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Initialized, Role: FOLLOWER
I20240502 14:00:31.391223 32316 consensus_queue.cc:260] T da2f3b3481da476d98c97bf35f055c2e P ad5befacd63742a1b5b69fdacdb8cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:31.391484 32316 ts_tablet_manager.cc:1398] T da2f3b3481da476d98c97bf35f055c2e P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:31.438349 32016 raft_consensus.cc:482] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:31.438477 32016 raft_consensus.cc:504] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:31.438860 32016 leader_election.cc:288] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553), 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:31.439143 30864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "954fa01ea4d04842b0218b33de347aa1" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" is_pre_election: true
I20240502 14:00:31.439141 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "954fa01ea4d04842b0218b33de347aa1" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929" is_pre_election: true
I20240502 14:00:31.439246 30864 raft_consensus.cc:2454] T 954fa01ea4d04842b0218b33de347aa1 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 0.
I20240502 14:00:31.439255 31003 raft_consensus.cc:2454] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 0.
I20240502 14:00:31.439543 31199 leader_election.cc:302] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:31.439735 32016 raft_consensus.cc:2781] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:31.439788 32016 raft_consensus.cc:482] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:31.439841 32016 raft_consensus.cc:3037] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.440644 32016 raft_consensus.cc:504] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:31.440970 32016 leader_election.cc:288] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 election: Requested vote from peers ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553), 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:31.441241 30864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "954fa01ea4d04842b0218b33de347aa1" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
I20240502 14:00:31.441265 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "954fa01ea4d04842b0218b33de347aa1" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
I20240502 14:00:31.441329 30864 raft_consensus.cc:3037] T 954fa01ea4d04842b0218b33de347aa1 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.441359 31003 raft_consensus.cc:3037] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.442018 30864 raft_consensus.cc:2454] T 954fa01ea4d04842b0218b33de347aa1 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 1.
I20240502 14:00:31.442063 31003 raft_consensus.cc:2454] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 1.
I20240502 14:00:31.442286 31199 leader_election.cc:302] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:31.442487 32016 raft_consensus.cc:2781] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:31.442579 32016 raft_consensus.cc:686] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Becoming Leader. State: Replica: 14753c93125941acaad7283edc669052, State: Running, Role: LEADER
I20240502 14:00:31.442714 32016 consensus_queue.cc:237] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:31.444212 30712 catalog_manager.cc:5484] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14753c93125941acaad7283edc669052 (127.24.66.68). New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:31.580144 32235 raft_consensus.cc:482] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:31.580284 32235 raft_consensus.cc:504] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:31.580704 32235 leader_election.cc:288] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14753c93125941acaad7283edc669052 (127.24.66.68:37029), 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:31.581125 31266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5b3cf5fbe3d4d999cbf910c8e0ca62e" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052" is_pre_election: true
I20240502 14:00:31.581127 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5b3cf5fbe3d4d999cbf910c8e0ca62e" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929" is_pre_election: true
I20240502 14:00:31.581243 31003 raft_consensus.cc:2454] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 0.
I20240502 14:00:31.581259 31266 raft_consensus.cc:2454] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 0.
I20240502 14:00:31.581564 30797 leader_election.cc:302] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fab10dcfb8542fca2b965fed6c67929, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:31.581734 32235 raft_consensus.cc:2781] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:31.581804 32235 raft_consensus.cc:482] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:31.581830 32235 raft_consensus.cc:3037] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.582638 32235 raft_consensus.cc:504] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:31.582926 32235 leader_election.cc:288] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 election: Requested vote from peers 14753c93125941acaad7283edc669052 (127.24.66.68:37029), 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:31.583248 31266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5b3cf5fbe3d4d999cbf910c8e0ca62e" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052"
I20240502 14:00:31.583266 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5b3cf5fbe3d4d999cbf910c8e0ca62e" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
I20240502 14:00:31.583339 31266 raft_consensus.cc:3037] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.583359 31003 raft_consensus.cc:3037] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.584036 31003 raft_consensus.cc:2454] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 1.
I20240502 14:00:31.584040 31266 raft_consensus.cc:2454] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 1.
I20240502 14:00:31.584270 30798 leader_election.cc:302] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:31.584473 32235 raft_consensus.cc:2781] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:31.584636 32235 raft_consensus.cc:686] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Becoming Leader. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Running, Role: LEADER
I20240502 14:00:31.584764 32235 consensus_queue.cc:237] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:31.586444 30712 catalog_manager.cc:5484] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: term changed from 0 to 1, leader changed from <none> to ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65). New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:31.742130 32016 raft_consensus.cc:482] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:31.742237 32016 raft_consensus.cc:504] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:31.742609 32016 leader_election.cc:288] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553), 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:31.742973 30864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da2f3b3481da476d98c97bf35f055c2e" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" is_pre_election: true
I20240502 14:00:31.742973 31133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da2f3b3481da476d98c97bf35f055c2e" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc" is_pre_election: true
I20240502 14:00:31.743113 31133 raft_consensus.cc:2454] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 0.
I20240502 14:00:31.743113 30864 raft_consensus.cc:2454] T da2f3b3481da476d98c97bf35f055c2e P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 0.
I20240502 14:00:31.743366 31202 leader_election.cc:302] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 8c61042437444f5f92c76109dfb229fc; no voters:
I20240502 14:00:31.743530 32120 raft_consensus.cc:2781] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:31.743604 32120 raft_consensus.cc:482] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:31.743634 32120 raft_consensus.cc:3037] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.744438 32120 raft_consensus.cc:504] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:31.744725 32120 leader_election.cc:288] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 election: Requested vote from peers ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553), 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:31.745011 30864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da2f3b3481da476d98c97bf35f055c2e" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
I20240502 14:00:31.745004 31133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da2f3b3481da476d98c97bf35f055c2e" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc"
I20240502 14:00:31.745112 31133 raft_consensus.cc:3037] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.745112 30864 raft_consensus.cc:3037] T da2f3b3481da476d98c97bf35f055c2e P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:31.745987 31133 raft_consensus.cc:2454] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 1.
I20240502 14:00:31.745986 30864 raft_consensus.cc:2454] T da2f3b3481da476d98c97bf35f055c2e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 1.
I20240502 14:00:31.746240 31199 leader_election.cc:302] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:31.746397 32120 raft_consensus.cc:2781] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:31.746481 32120 raft_consensus.cc:686] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Becoming Leader. State: Replica: 14753c93125941acaad7283edc669052, State: Running, Role: LEADER
I20240502 14:00:31.746609 32120 consensus_queue.cc:237] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:31.748139 30712 catalog_manager.cc:5484] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14753c93125941acaad7283edc669052 (127.24.66.68). New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:31.856585 32120 consensus_queue.cc:1035] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:31.859921 32120 consensus_queue.cc:1035] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 14:00:31.920100 32324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:31.920348 32324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:31.923184 32324 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_6'
I20240502 14:00:31.965992 32235 consensus_queue.cc:1035] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:31.968932 32321 consensus_queue.cc:1035] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:32.030956 31134 raft_consensus.cc:1261] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:32.031036 30865 raft_consensus.cc:1261] T da2f3b3481da476d98c97bf35f055c2e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:32.031368 32016 consensus_queue.cc:1035] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:32.031622 32016 consensus_queue.cc:1035] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 101.523 ms
time per row: 0.0507617 ms
I20240502 14:00:32.065145 30711 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38902:
name: "rebalance_test_table_7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:32.069772 31115 tablet_service.cc:1467] Processing CreateTablet for tablet 8da28cc4aef043baaa2282f84afd4abf (DEFAULT_TABLE table=rebalance_test_table_7 [id=2c4adac4a0a74af9bacbd4987a8a8d39]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.069978 30982 tablet_service.cc:1467] Processing CreateTablet for tablet 8da28cc4aef043baaa2282f84afd4abf (DEFAULT_TABLE table=rebalance_test_table_7 [id=2c4adac4a0a74af9bacbd4987a8a8d39]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.070048 31115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8da28cc4aef043baaa2282f84afd4abf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.070144 30845 tablet_service.cc:1467] Processing CreateTablet for tablet d0409950632d4eeead8a826034b5a9e4 (DEFAULT_TABLE table=rebalance_test_table_7 [id=2c4adac4a0a74af9bacbd4987a8a8d39]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.070228 30982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8da28cc4aef043baaa2282f84afd4abf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.070375 30845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0409950632d4eeead8a826034b5a9e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.070498 30843 tablet_service.cc:1467] Processing CreateTablet for tablet 8da28cc4aef043baaa2282f84afd4abf (DEFAULT_TABLE table=rebalance_test_table_7 [id=2c4adac4a0a74af9bacbd4987a8a8d39]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.070710 30843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8da28cc4aef043baaa2282f84afd4abf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.072387 32383 tablet_bootstrap.cc:492] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc: Bootstrap starting.
I20240502 14:00:32.073194 31244 tablet_service.cc:1467] Processing CreateTablet for tablet d0409950632d4eeead8a826034b5a9e4 (DEFAULT_TABLE table=rebalance_test_table_7 [id=2c4adac4a0a74af9bacbd4987a8a8d39]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.073220 31246 tablet_service.cc:1467] Processing CreateTablet for tablet c0b9178646764824b17cbd842fbb0ff2 (DEFAULT_TABLE table=rebalance_test_table_7 [id=2c4adac4a0a74af9bacbd4987a8a8d39]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.073220 31115 tablet_service.cc:1467] Processing CreateTablet for tablet d0409950632d4eeead8a826034b5a9e4 (DEFAULT_TABLE table=rebalance_test_table_7 [id=2c4adac4a0a74af9bacbd4987a8a8d39]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.073436 31115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0409950632d4eeead8a826034b5a9e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.073451 31244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0409950632d4eeead8a826034b5a9e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.073459 32383 tablet_bootstrap.cc:654] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.073632 31246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0b9178646764824b17cbd842fbb0ff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.075256 30982 tablet_service.cc:1467] Processing CreateTablet for tablet c0b9178646764824b17cbd842fbb0ff2 (DEFAULT_TABLE table=rebalance_test_table_7 [id=2c4adac4a0a74af9bacbd4987a8a8d39]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.075378 31113 tablet_service.cc:1467] Processing CreateTablet for tablet c0b9178646764824b17cbd842fbb0ff2 (DEFAULT_TABLE table=rebalance_test_table_7 [id=2c4adac4a0a74af9bacbd4987a8a8d39]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.075479 30982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0b9178646764824b17cbd842fbb0ff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.075585 31113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0b9178646764824b17cbd842fbb0ff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.075695 32385 tablet_bootstrap.cc:492] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f: Bootstrap starting.
I20240502 14:00:32.076151 32387 tablet_bootstrap.cc:492] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052: Bootstrap starting.
I20240502 14:00:32.076736 32385 tablet_bootstrap.cc:654] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.077179 32387 tablet_bootstrap.cc:654] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.077839 32385 tablet_bootstrap.cc:492] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f: No bootstrap required, opened a new log
I20240502 14:00:32.077838 32383 tablet_bootstrap.cc:492] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc: No bootstrap required, opened a new log
I20240502 14:00:32.077920 32385 ts_tablet_manager.cc:1367] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:32.077920 32383 ts_tablet_manager.cc:1367] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 14:00:32.078037 32384 tablet_bootstrap.cc:492] T 8da28cc4aef043baaa2282f84afd4abf P 1fab10dcfb8542fca2b965fed6c67929: Bootstrap starting.
I20240502 14:00:32.078295 32383 raft_consensus.cc:348] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:32.078295 32385 raft_consensus.cc:348] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:32.078401 32383 raft_consensus.cc:374] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.078401 32385 raft_consensus.cc:374] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.078441 32383 raft_consensus.cc:729] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.078446 32385 raft_consensus.cc:729] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.078572 32383 consensus_queue.cc:260] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:32.078572 32385 consensus_queue.cc:260] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:32.078825 32383 ts_tablet_manager.cc:1398] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:32.078825 32385 ts_tablet_manager.cc:1398] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:32.078970 32383 tablet_bootstrap.cc:492] T d0409950632d4eeead8a826034b5a9e4 P 8c61042437444f5f92c76109dfb229fc: Bootstrap starting.
I20240502 14:00:32.078970 32385 tablet_bootstrap.cc:492] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f: Bootstrap starting.
I20240502 14:00:32.079344 32384 tablet_bootstrap.cc:654] T 8da28cc4aef043baaa2282f84afd4abf P 1fab10dcfb8542fca2b965fed6c67929: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.079680 32383 tablet_bootstrap.cc:654] T d0409950632d4eeead8a826034b5a9e4 P 8c61042437444f5f92c76109dfb229fc: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.079722 32385 tablet_bootstrap.cc:654] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.080358 32384 tablet_bootstrap.cc:492] T 8da28cc4aef043baaa2282f84afd4abf P 1fab10dcfb8542fca2b965fed6c67929: No bootstrap required, opened a new log
I20240502 14:00:32.080440 32384 ts_tablet_manager.cc:1367] T 8da28cc4aef043baaa2282f84afd4abf P 1fab10dcfb8542fca2b965fed6c67929: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:32.080811 32384 raft_consensus.cc:348] T 8da28cc4aef043baaa2282f84afd4abf P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:32.080935 32384 raft_consensus.cc:374] T 8da28cc4aef043baaa2282f84afd4abf P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.080969 32384 raft_consensus.cc:729] T 8da28cc4aef043baaa2282f84afd4abf P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.081331 32384 consensus_queue.cc:260] T 8da28cc4aef043baaa2282f84afd4abf P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:32.081905 32385 tablet_bootstrap.cc:492] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f: No bootstrap required, opened a new log
I20240502 14:00:32.081972 32385 ts_tablet_manager.cc:1367] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:32.082002 32383 tablet_bootstrap.cc:492] T d0409950632d4eeead8a826034b5a9e4 P 8c61042437444f5f92c76109dfb229fc: No bootstrap required, opened a new log
I20240502 14:00:32.082063 32383 ts_tablet_manager.cc:1367] T d0409950632d4eeead8a826034b5a9e4 P 8c61042437444f5f92c76109dfb229fc: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:32.082387 32384 ts_tablet_manager.cc:1398] T 8da28cc4aef043baaa2282f84afd4abf P 1fab10dcfb8542fca2b965fed6c67929: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:32.082502 32383 raft_consensus.cc:348] T d0409950632d4eeead8a826034b5a9e4 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:32.082391 32385 raft_consensus.cc:348] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:32.082612 32383 raft_consensus.cc:374] T d0409950632d4eeead8a826034b5a9e4 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.082619 32385 raft_consensus.cc:374] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.082643 32384 tablet_bootstrap.cc:492] T c0b9178646764824b17cbd842fbb0ff2 P 1fab10dcfb8542fca2b965fed6c67929: Bootstrap starting.
I20240502 14:00:32.082664 32383 raft_consensus.cc:729] T d0409950632d4eeead8a826034b5a9e4 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.082664 32385 raft_consensus.cc:729] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.082780 32385 consensus_queue.cc:260] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:32.082780 32383 consensus_queue.cc:260] T d0409950632d4eeead8a826034b5a9e4 P 8c61042437444f5f92c76109dfb229fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:32.083034 32383 ts_tablet_manager.cc:1398] T d0409950632d4eeead8a826034b5a9e4 P 8c61042437444f5f92c76109dfb229fc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:32.083034 32385 ts_tablet_manager.cc:1398] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:32.083518 32383 tablet_bootstrap.cc:492] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc: Bootstrap starting.
I20240502 14:00:32.084151 32384 tablet_bootstrap.cc:654] T c0b9178646764824b17cbd842fbb0ff2 P 1fab10dcfb8542fca2b965fed6c67929: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.084434 32383 tablet_bootstrap.cc:654] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.085860 32387 tablet_bootstrap.cc:492] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052: No bootstrap required, opened a new log
I20240502 14:00:32.085947 32387 ts_tablet_manager.cc:1367] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 14:00:32.086388 32387 raft_consensus.cc:348] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:32.086479 32387 raft_consensus.cc:374] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.086515 32387 raft_consensus.cc:729] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14753c93125941acaad7283edc669052, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.086621 32387 consensus_queue.cc:260] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:32.086890 32387 ts_tablet_manager.cc:1398] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:32.087061 32387 tablet_bootstrap.cc:492] T c0b9178646764824b17cbd842fbb0ff2 P 14753c93125941acaad7283edc669052: Bootstrap starting.
I20240502 14:00:32.087800 32387 tablet_bootstrap.cc:654] T c0b9178646764824b17cbd842fbb0ff2 P 14753c93125941acaad7283edc669052: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.088826 32383 tablet_bootstrap.cc:492] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc: No bootstrap required, opened a new log
I20240502 14:00:32.088833 32387 tablet_bootstrap.cc:492] T c0b9178646764824b17cbd842fbb0ff2 P 14753c93125941acaad7283edc669052: No bootstrap required, opened a new log
I20240502 14:00:32.088907 32383 ts_tablet_manager.cc:1367] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240502 14:00:32.088907 32387 ts_tablet_manager.cc:1367] T c0b9178646764824b17cbd842fbb0ff2 P 14753c93125941acaad7283edc669052: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:32.089301 32383 raft_consensus.cc:348] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:32.089301 32387 raft_consensus.cc:348] T c0b9178646764824b17cbd842fbb0ff2 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:32.089406 32383 raft_consensus.cc:374] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.089406 32387 raft_consensus.cc:374] T c0b9178646764824b17cbd842fbb0ff2 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.089437 32383 raft_consensus.cc:729] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.089437 32387 raft_consensus.cc:729] T c0b9178646764824b17cbd842fbb0ff2 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14753c93125941acaad7283edc669052, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.089555 32383 consensus_queue.cc:260] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:32.089589 32387 consensus_queue.cc:260] T c0b9178646764824b17cbd842fbb0ff2 P 14753c93125941acaad7283edc669052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:32.089809 32387 ts_tablet_manager.cc:1398] T c0b9178646764824b17cbd842fbb0ff2 P 14753c93125941acaad7283edc669052: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:32.089833 32383 ts_tablet_manager.cc:1398] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:32.090457 32384 tablet_bootstrap.cc:492] T c0b9178646764824b17cbd842fbb0ff2 P 1fab10dcfb8542fca2b965fed6c67929: No bootstrap required, opened a new log
I20240502 14:00:32.090535 32384 ts_tablet_manager.cc:1367] T c0b9178646764824b17cbd842fbb0ff2 P 1fab10dcfb8542fca2b965fed6c67929: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:00:32.091060 32384 raft_consensus.cc:348] T c0b9178646764824b17cbd842fbb0ff2 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:32.091153 32384 raft_consensus.cc:374] T c0b9178646764824b17cbd842fbb0ff2 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.091183 32384 raft_consensus.cc:729] T c0b9178646764824b17cbd842fbb0ff2 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.091347 32384 consensus_queue.cc:260] T c0b9178646764824b17cbd842fbb0ff2 P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:32.091660 32384 ts_tablet_manager.cc:1398] T c0b9178646764824b17cbd842fbb0ff2 P 1fab10dcfb8542fca2b965fed6c67929: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:32.161091 32321 raft_consensus.cc:482] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:32.161232 32321 raft_consensus.cc:504] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:32.161777 32321 leader_election.cc:288] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193), 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:32.162043 31134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0409950632d4eeead8a826034b5a9e4" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc" is_pre_election: true
I20240502 14:00:32.162194 31134 raft_consensus.cc:2454] T d0409950632d4eeead8a826034b5a9e4 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 0.
I20240502 14:00:32.162480 30798 leader_election.cc:302] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c61042437444f5f92c76109dfb229fc, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:32.162659 32321 raft_consensus.cc:2781] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:32.162617 31266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0409950632d4eeead8a826034b5a9e4" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052" is_pre_election: true
I20240502 14:00:32.162719 32321 raft_consensus.cc:482] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:32.162757 32321 raft_consensus.cc:3037] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.162765 31266 raft_consensus.cc:2454] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 0.
I20240502 14:00:32.163645 32321 raft_consensus.cc:504] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:32.163955 32321 leader_election.cc:288] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 election: Requested vote from peers 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193), 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:32.164268 31266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0409950632d4eeead8a826034b5a9e4" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052"
I20240502 14:00:32.164407 31266 raft_consensus.cc:3037] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.165154 31266 raft_consensus.cc:2454] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 1.
I20240502 14:00:32.165360 31134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0409950632d4eeead8a826034b5a9e4" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc"
I20240502 14:00:32.165428 30798 leader_election.cc:302] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:32.165475 31134 raft_consensus.cc:3037] T d0409950632d4eeead8a826034b5a9e4 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.165585 32321 raft_consensus.cc:2781] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:32.165685 32321 raft_consensus.cc:686] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Becoming Leader. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Running, Role: LEADER
I20240502 14:00:32.165810 32321 consensus_queue.cc:237] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:32.166388 31134 raft_consensus.cc:2454] T d0409950632d4eeead8a826034b5a9e4 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 1.
I20240502 14:00:32.167897 30711 catalog_manager.cc:5484] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: term changed from 0 to 1, leader changed from <none> to ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65). New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:32.229902 32321 raft_consensus.cc:482] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:32.230017 32321 raft_consensus.cc:504] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:32.230515 32321 leader_election.cc:288] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193), 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:32.230762 31134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8da28cc4aef043baaa2282f84afd4abf" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc" is_pre_election: true
I20240502 14:00:32.230880 31134 raft_consensus.cc:2454] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 0.
I20240502 14:00:32.230894 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8da28cc4aef043baaa2282f84afd4abf" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929" is_pre_election: true
I20240502 14:00:32.231007 31003 raft_consensus.cc:2454] T 8da28cc4aef043baaa2282f84afd4abf P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 0.
I20240502 14:00:32.231155 30798 leader_election.cc:302] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c61042437444f5f92c76109dfb229fc, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:32.231288 32321 raft_consensus.cc:2781] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:32.231357 32321 raft_consensus.cc:482] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:32.231402 32321 raft_consensus.cc:3037] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.232229 32321 raft_consensus.cc:504] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:32.232573 32321 leader_election.cc:288] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 election: Requested vote from peers 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193), 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:32.232834 31134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8da28cc4aef043baaa2282f84afd4abf" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc"
I20240502 14:00:32.232942 31134 raft_consensus.cc:3037] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.232964 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8da28cc4aef043baaa2282f84afd4abf" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
I20240502 14:00:32.233068 31003 raft_consensus.cc:3037] T 8da28cc4aef043baaa2282f84afd4abf P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.233814 31134 raft_consensus.cc:2454] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 1.
I20240502 14:00:32.233994 31003 raft_consensus.cc:2454] T 8da28cc4aef043baaa2282f84afd4abf P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 1.
I20240502 14:00:32.234277 30798 leader_election.cc:302] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c61042437444f5f92c76109dfb229fc, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:32.234597 32321 raft_consensus.cc:2781] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:32.234716 32321 raft_consensus.cc:686] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Becoming Leader. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Running, Role: LEADER
I20240502 14:00:32.234897 32321 consensus_queue.cc:237] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:32.236645 30711 catalog_manager.cc:5484] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: term changed from 0 to 1, leader changed from <none> to ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65). New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:32.470114 32381 raft_consensus.cc:482] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:32.470196 32381 raft_consensus.cc:504] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:32.470494 32381 leader_election.cc:288] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395), 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:32.470813 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b9178646764824b17cbd842fbb0ff2" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929" is_pre_election: true
I20240502 14:00:32.470801 31266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b9178646764824b17cbd842fbb0ff2" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052" is_pre_election: true
I20240502 14:00:32.470923 31003 raft_consensus.cc:2454] T c0b9178646764824b17cbd842fbb0ff2 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 0.
I20240502 14:00:32.470923 31266 raft_consensus.cc:2454] T c0b9178646764824b17cbd842fbb0ff2 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 0.
I20240502 14:00:32.471141 31070 leader_election.cc:302] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 8c61042437444f5f92c76109dfb229fc; no voters:
I20240502 14:00:32.471293 32381 raft_consensus.cc:2781] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:32.471350 32381 raft_consensus.cc:482] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:32.471375 32381 raft_consensus.cc:3037] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.472084 32381 raft_consensus.cc:504] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:32.472394 32381 leader_election.cc:288] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 1 election: Requested vote from peers 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395), 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:32.472658 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b9178646764824b17cbd842fbb0ff2" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
I20240502 14:00:32.472662 31266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0b9178646764824b17cbd842fbb0ff2" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052"
I20240502 14:00:32.472760 31003 raft_consensus.cc:3037] T c0b9178646764824b17cbd842fbb0ff2 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.472760 31266 raft_consensus.cc:3037] T c0b9178646764824b17cbd842fbb0ff2 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.473659 31003 raft_consensus.cc:2454] T c0b9178646764824b17cbd842fbb0ff2 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 1.
I20240502 14:00:32.473663 31266 raft_consensus.cc:2454] T c0b9178646764824b17cbd842fbb0ff2 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 1.
I20240502 14:00:32.473906 31070 leader_election.cc:302] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 8c61042437444f5f92c76109dfb229fc; no voters:
I20240502 14:00:32.474046 32381 raft_consensus.cc:2781] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:32.474124 32381 raft_consensus.cc:686] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Becoming Leader. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Running, Role: LEADER
I20240502 14:00:32.474260 32381 consensus_queue.cc:237] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:32.475805 30711 catalog_manager.cc:5484] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c61042437444f5f92c76109dfb229fc (127.24.66.67). New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:32.571926 32321 consensus_queue.cc:1035] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:32.574961 32321 consensus_queue.cc:1035] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 14:00:32.637670 32399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:32.637879 32399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:32.640672 32399 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_7'
I20240502 14:00:32.718297 31003 raft_consensus.cc:1261] T c0b9178646764824b17cbd842fbb0ff2 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:32.719030 32381 consensus_queue.cc:1035] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:32.719537 31266 raft_consensus.cc:1261] T c0b9178646764824b17cbd842fbb0ff2 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:32.719687 31134 raft_consensus.cc:1261] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:32.720078 31002 raft_consensus.cc:1261] T 8da28cc4aef043baaa2282f84afd4abf P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:32.720405 32381 consensus_queue.cc:1035] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:32.720469 32235 consensus_queue.cc:1035] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:32.724565 32321 consensus_queue.cc:1035] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 94.9123 ms
time per row: 0.0474561 ms
I20240502 14:00:32.774312 30710 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38902:
name: "rebalance_test_table_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:32.779299 30843 tablet_service.cc:1467] Processing CreateTablet for tablet 4a12b8eaab7943c3bc9ee48c2a2307d0 (DEFAULT_TABLE table=rebalance_test_table_8 [id=5cb834cd98904017a28fc7b595efc564]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.779395 30982 tablet_service.cc:1467] Processing CreateTablet for tablet 4a12b8eaab7943c3bc9ee48c2a2307d0 (DEFAULT_TABLE table=rebalance_test_table_8 [id=5cb834cd98904017a28fc7b595efc564]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.779600 30843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a12b8eaab7943c3bc9ee48c2a2307d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.779587 31244 tablet_service.cc:1467] Processing CreateTablet for tablet 4a12b8eaab7943c3bc9ee48c2a2307d0 (DEFAULT_TABLE table=rebalance_test_table_8 [id=5cb834cd98904017a28fc7b595efc564]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.779671 30982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a12b8eaab7943c3bc9ee48c2a2307d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.779821 31244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a12b8eaab7943c3bc9ee48c2a2307d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.780273 31246 tablet_service.cc:1467] Processing CreateTablet for tablet 5a3c9c93437c43b1a9c0326f2880a04e (DEFAULT_TABLE table=rebalance_test_table_8 [id=5cb834cd98904017a28fc7b595efc564]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.780498 31246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a3c9c93437c43b1a9c0326f2880a04e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.782121 32458 tablet_bootstrap.cc:492] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f: Bootstrap starting.
I20240502 14:00:32.782114 32460 tablet_bootstrap.cc:492] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 14753c93125941acaad7283edc669052: Bootstrap starting.
I20240502 14:00:32.782883 30843 tablet_service.cc:1467] Processing CreateTablet for tablet 2e07f0adca594453bad9433ed0017580 (DEFAULT_TABLE table=rebalance_test_table_8 [id=5cb834cd98904017a28fc7b595efc564]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.782873 30982 tablet_service.cc:1467] Processing CreateTablet for tablet 2e07f0adca594453bad9433ed0017580 (DEFAULT_TABLE table=rebalance_test_table_8 [id=5cb834cd98904017a28fc7b595efc564]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.783078 30843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e07f0adca594453bad9433ed0017580. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.783079 30982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e07f0adca594453bad9433ed0017580. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.783138 32458 tablet_bootstrap.cc:654] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.783159 32460 tablet_bootstrap.cc:654] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 14753c93125941acaad7283edc669052: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.783531 30845 tablet_service.cc:1467] Processing CreateTablet for tablet 5a3c9c93437c43b1a9c0326f2880a04e (DEFAULT_TABLE table=rebalance_test_table_8 [id=5cb834cd98904017a28fc7b595efc564]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.783743 30845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a3c9c93437c43b1a9c0326f2880a04e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.784186 32458 tablet_bootstrap.cc:492] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f: No bootstrap required, opened a new log
I20240502 14:00:32.784242 32458 ts_tablet_manager.cc:1367] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:32.784704 32458 raft_consensus.cc:348] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:32.784809 32458 raft_consensus.cc:374] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.784842 32458 raft_consensus.cc:729] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.784956 32458 consensus_queue.cc:260] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:32.785274 32458 ts_tablet_manager.cc:1398] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:32.785629 32458 tablet_bootstrap.cc:492] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f: Bootstrap starting.
I20240502 14:00:32.786024 32460 tablet_bootstrap.cc:492] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 14753c93125941acaad7283edc669052: No bootstrap required, opened a new log
I20240502 14:00:32.786134 32460 ts_tablet_manager.cc:1367] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 14753c93125941acaad7283edc669052: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:00:32.786453 32458 tablet_bootstrap.cc:654] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.786461 31113 tablet_service.cc:1467] Processing CreateTablet for tablet 2e07f0adca594453bad9433ed0017580 (DEFAULT_TABLE table=rebalance_test_table_8 [id=5cb834cd98904017a28fc7b595efc564]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.786556 32460 raft_consensus.cc:348] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:32.786650 32460 raft_consensus.cc:374] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.786696 32460 raft_consensus.cc:729] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14753c93125941acaad7283edc669052, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.786717 31113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e07f0adca594453bad9433ed0017580. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.786834 32460 consensus_queue.cc:260] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 14753c93125941acaad7283edc669052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:32.787128 32460 ts_tablet_manager.cc:1398] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 14753c93125941acaad7283edc669052: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:32.787292 32460 tablet_bootstrap.cc:492] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052: Bootstrap starting.
I20240502 14:00:32.788129 32460 tablet_bootstrap.cc:654] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.788695 31115 tablet_service.cc:1467] Processing CreateTablet for tablet 5a3c9c93437c43b1a9c0326f2880a04e (DEFAULT_TABLE table=rebalance_test_table_8 [id=5cb834cd98904017a28fc7b595efc564]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:32.788918 31115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a3c9c93437c43b1a9c0326f2880a04e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:32.789216 32460 tablet_bootstrap.cc:492] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052: No bootstrap required, opened a new log
I20240502 14:00:32.789296 32460 ts_tablet_manager.cc:1367] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:32.789603 32460 raft_consensus.cc:348] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:32.789697 32460 raft_consensus.cc:374] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.789741 32460 raft_consensus.cc:729] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14753c93125941acaad7283edc669052, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.789862 32460 consensus_queue.cc:260] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:32.790112 32460 ts_tablet_manager.cc:1398] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:32.792517 32464 tablet_bootstrap.cc:492] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc: Bootstrap starting.
I20240502 14:00:32.793292 32464 tablet_bootstrap.cc:654] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.793802 32459 tablet_bootstrap.cc:492] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 1fab10dcfb8542fca2b965fed6c67929: Bootstrap starting.
I20240502 14:00:32.794291 32464 tablet_bootstrap.cc:492] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc: No bootstrap required, opened a new log
I20240502 14:00:32.794412 32464 ts_tablet_manager.cc:1367] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:32.794574 32459 tablet_bootstrap.cc:654] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 1fab10dcfb8542fca2b965fed6c67929: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.794862 32464 raft_consensus.cc:348] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:32.794945 32464 raft_consensus.cc:374] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.794979 32464 raft_consensus.cc:729] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.795133 32464 consensus_queue.cc:260] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:32.795425 32464 ts_tablet_manager.cc:1398] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:32.795945 32459 tablet_bootstrap.cc:492] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 1fab10dcfb8542fca2b965fed6c67929: No bootstrap required, opened a new log
I20240502 14:00:32.796022 32459 ts_tablet_manager.cc:1367] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 1fab10dcfb8542fca2b965fed6c67929: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:32.796509 32464 tablet_bootstrap.cc:492] T 5a3c9c93437c43b1a9c0326f2880a04e P 8c61042437444f5f92c76109dfb229fc: Bootstrap starting.
I20240502 14:00:32.796485 32459 raft_consensus.cc:348] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:32.796577 32459 raft_consensus.cc:374] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.796660 32458 tablet_bootstrap.cc:492] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f: No bootstrap required, opened a new log
I20240502 14:00:32.796675 32459 raft_consensus.cc:729] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.796736 32458 ts_tablet_manager.cc:1367] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
I20240502 14:00:32.796820 32459 consensus_queue.cc:260] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:32.797143 32459 ts_tablet_manager.cc:1398] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 1fab10dcfb8542fca2b965fed6c67929: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:32.797129 32458 raft_consensus.cc:348] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:32.797223 32458 raft_consensus.cc:374] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.797255 32458 raft_consensus.cc:729] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.797324 32459 tablet_bootstrap.cc:492] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929: Bootstrap starting.
I20240502 14:00:32.797396 32458 consensus_queue.cc:260] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:32.797593 32464 tablet_bootstrap.cc:654] T 5a3c9c93437c43b1a9c0326f2880a04e P 8c61042437444f5f92c76109dfb229fc: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.797672 32458 ts_tablet_manager.cc:1398] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:32.797822 32458 tablet_bootstrap.cc:492] T 2e07f0adca594453bad9433ed0017580 P ad5befacd63742a1b5b69fdacdb8cf8f: Bootstrap starting.
I20240502 14:00:32.798348 32459 tablet_bootstrap.cc:654] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.799470 32458 tablet_bootstrap.cc:654] T 2e07f0adca594453bad9433ed0017580 P ad5befacd63742a1b5b69fdacdb8cf8f: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:32.800592 32458 tablet_bootstrap.cc:492] T 2e07f0adca594453bad9433ed0017580 P ad5befacd63742a1b5b69fdacdb8cf8f: No bootstrap required, opened a new log
I20240502 14:00:32.800669 32458 ts_tablet_manager.cc:1367] T 2e07f0adca594453bad9433ed0017580 P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 14:00:32.800927 32459 tablet_bootstrap.cc:492] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929: No bootstrap required, opened a new log
I20240502 14:00:32.800992 32459 ts_tablet_manager.cc:1367] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:00:32.801033 32464 tablet_bootstrap.cc:492] T 5a3c9c93437c43b1a9c0326f2880a04e P 8c61042437444f5f92c76109dfb229fc: No bootstrap required, opened a new log
I20240502 14:00:32.801108 32464 ts_tablet_manager.cc:1367] T 5a3c9c93437c43b1a9c0326f2880a04e P 8c61042437444f5f92c76109dfb229fc: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:00:32.801055 32458 raft_consensus.cc:348] T 2e07f0adca594453bad9433ed0017580 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:32.801178 32458 raft_consensus.cc:374] T 2e07f0adca594453bad9433ed0017580 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.801215 32458 raft_consensus.cc:729] T 2e07f0adca594453bad9433ed0017580 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.801338 32458 consensus_queue.cc:260] T 2e07f0adca594453bad9433ed0017580 P ad5befacd63742a1b5b69fdacdb8cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:32.801565 32458 ts_tablet_manager.cc:1398] T 2e07f0adca594453bad9433ed0017580 P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:32.801559 32459 raft_consensus.cc:348] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:32.801666 32459 raft_consensus.cc:374] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.801621 32464 raft_consensus.cc:348] T 5a3c9c93437c43b1a9c0326f2880a04e P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:32.801702 32459 raft_consensus.cc:729] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.801726 32464 raft_consensus.cc:374] T 5a3c9c93437c43b1a9c0326f2880a04e P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:32.801788 32464 raft_consensus.cc:729] T 5a3c9c93437c43b1a9c0326f2880a04e P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Initialized, Role: FOLLOWER
I20240502 14:00:32.801851 32459 consensus_queue.cc:260] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:32.801919 32464 consensus_queue.cc:260] T 5a3c9c93437c43b1a9c0326f2880a04e P 8c61042437444f5f92c76109dfb229fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:32.802191 32459 ts_tablet_manager.cc:1398] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:32.803350 32464 ts_tablet_manager.cc:1398] T 5a3c9c93437c43b1a9c0326f2880a04e P 8c61042437444f5f92c76109dfb229fc: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:32.820377 32381 raft_consensus.cc:482] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:32.820469 32381 raft_consensus.cc:504] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:32.820787 32381 leader_election.cc:288] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553), 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:32.821079 30865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e07f0adca594453bad9433ed0017580" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" is_pre_election: true
I20240502 14:00:32.821130 31002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e07f0adca594453bad9433ed0017580" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929" is_pre_election: true
I20240502 14:00:32.821187 30865 raft_consensus.cc:2454] T 2e07f0adca594453bad9433ed0017580 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 0.
I20240502 14:00:32.821229 31002 raft_consensus.cc:2454] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 0.
I20240502 14:00:32.821501 31067 leader_election.cc:302] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c61042437444f5f92c76109dfb229fc, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:32.821645 32381 raft_consensus.cc:2781] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:32.821698 32381 raft_consensus.cc:482] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:32.821735 32381 raft_consensus.cc:3037] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.822365 32381 raft_consensus.cc:504] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:32.822623 32381 leader_election.cc:288] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 1 election: Requested vote from peers ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553), 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:32.822902 30865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e07f0adca594453bad9433ed0017580" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
I20240502 14:00:32.822995 30865 raft_consensus.cc:3037] T 2e07f0adca594453bad9433ed0017580 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.823016 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e07f0adca594453bad9433ed0017580" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
I20240502 14:00:32.823101 31003 raft_consensus.cc:3037] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.823748 30865 raft_consensus.cc:2454] T 2e07f0adca594453bad9433ed0017580 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 1.
I20240502 14:00:32.823799 31003 raft_consensus.cc:2454] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 1.
I20240502 14:00:32.824111 31067 leader_election.cc:302] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c61042437444f5f92c76109dfb229fc, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:32.824268 32250 raft_consensus.cc:2781] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:32.824371 32250 raft_consensus.cc:686] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Becoming Leader. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Running, Role: LEADER
I20240502 14:00:32.824503 32250 consensus_queue.cc:237] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } }
I20240502 14:00:32.825973 30715 catalog_manager.cc:5484] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c61042437444f5f92c76109dfb229fc (127.24.66.67). New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:32.900807 32436 raft_consensus.cc:482] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:32.900894 32436 raft_consensus.cc:504] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:32.901170 32436 leader_election.cc:288] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14753c93125941acaad7283edc669052 (127.24.66.68:37029), 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:32.901487 31266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3c9c93437c43b1a9c0326f2880a04e" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052" is_pre_election: true
I20240502 14:00:32.901530 31134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3c9c93437c43b1a9c0326f2880a04e" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc" is_pre_election: true
I20240502 14:00:32.901600 31266 raft_consensus.cc:2454] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 0.
I20240502 14:00:32.901670 31134 raft_consensus.cc:2454] T 5a3c9c93437c43b1a9c0326f2880a04e P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 0.
I20240502 14:00:32.901870 30798 leader_election.cc:302] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:32.902014 32436 raft_consensus.cc:2781] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:32.902066 32436 raft_consensus.cc:482] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:32.902109 32436 raft_consensus.cc:3037] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.902814 32436 raft_consensus.cc:504] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:32.903060 32436 leader_election.cc:288] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 election: Requested vote from peers 14753c93125941acaad7283edc669052 (127.24.66.68:37029), 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:32.903352 31266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3c9c93437c43b1a9c0326f2880a04e" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052"
I20240502 14:00:32.903393 31134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3c9c93437c43b1a9c0326f2880a04e" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc"
I20240502 14:00:32.903470 31266 raft_consensus.cc:3037] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.903481 31134 raft_consensus.cc:3037] T 5a3c9c93437c43b1a9c0326f2880a04e P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:32.904309 31266 raft_consensus.cc:2454] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 1.
I20240502 14:00:32.904309 31134 raft_consensus.cc:2454] T 5a3c9c93437c43b1a9c0326f2880a04e P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 1.
I20240502 14:00:32.904597 30798 leader_election.cc:302] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:32.904754 32436 raft_consensus.cc:2781] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:32.904840 32436 raft_consensus.cc:686] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Becoming Leader. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Running, Role: LEADER
I20240502 14:00:32.904971 32436 consensus_queue.cc:237] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:32.906503 30715 catalog_manager.cc:5484] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: term changed from 0 to 1, leader changed from <none> to ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65). New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:33.065935 32436 raft_consensus.cc:482] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:33.066030 32436 raft_consensus.cc:504] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:33.066349 32436 leader_election.cc:288] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395), 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:33.066601 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a12b8eaab7943c3bc9ee48c2a2307d0" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929" is_pre_election: true
I20240502 14:00:33.066668 31266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a12b8eaab7943c3bc9ee48c2a2307d0" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052" is_pre_election: true
I20240502 14:00:33.066711 31003 raft_consensus.cc:2454] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 0.
I20240502 14:00:33.066802 31266 raft_consensus.cc:2454] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 0.
I20240502 14:00:33.066960 30797 leader_election.cc:302] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fab10dcfb8542fca2b965fed6c67929, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:33.067121 32436 raft_consensus.cc:2781] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:33.067179 32436 raft_consensus.cc:482] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:33.067204 32436 raft_consensus.cc:3037] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:33.067924 32436 raft_consensus.cc:504] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:33.068197 32436 leader_election.cc:288] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 election: Requested vote from peers 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395), 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:33.068542 31266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a12b8eaab7943c3bc9ee48c2a2307d0" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052"
I20240502 14:00:33.068536 31003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a12b8eaab7943c3bc9ee48c2a2307d0" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
I20240502 14:00:33.068647 31003 raft_consensus.cc:3037] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:33.068647 31266 raft_consensus.cc:3037] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:33.069468 31266 raft_consensus.cc:2454] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 1.
I20240502 14:00:33.069609 31003 raft_consensus.cc:2454] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 1.
I20240502 14:00:33.069743 30798 leader_election.cc:302] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:33.069906 32436 raft_consensus.cc:2781] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:33.069988 32436 raft_consensus.cc:686] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Becoming Leader. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Running, Role: LEADER
I20240502 14:00:33.070132 32436 consensus_queue.cc:237] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:33.071779 30712 catalog_manager.cc:5484] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: term changed from 0 to 1, leader changed from <none> to ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65). New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:33.244302 32474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:33.244550 32474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:33.247401 32474 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_8'
I20240502 14:00:33.303411 32381 consensus_queue.cc:1035] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:33.312645 32250 consensus_queue.cc:1035] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:33.336519 31266 raft_consensus.cc:1261] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:33.336985 32235 consensus_queue.cc:1035] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:33.337657 31267 raft_consensus.cc:1261] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:33.337950 31003 raft_consensus.cc:1261] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:33.338100 32436 consensus_queue.cc:1035] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:33.339133 31134 raft_consensus.cc:1261] T 5a3c9c93437c43b1a9c0326f2880a04e P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:33.339783 32436 consensus_queue.cc:1035] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:33.340057 32436 consensus_queue.cc:1035] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 124.109 ms
time per row: 0.0620547 ms
I20240502 14:00:33.409492 30715 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38902:
name: "rebalance_test_table_9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:33.414624 30982 tablet_service.cc:1467] Processing CreateTablet for tablet c43ff8fc0b4f4f799a6f1bbd677166da (DEFAULT_TABLE table=rebalance_test_table_9 [id=5f28187b5b7a472f8429ed627f138f76]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:33.414639 31115 tablet_service.cc:1467] Processing CreateTablet for tablet c43ff8fc0b4f4f799a6f1bbd677166da (DEFAULT_TABLE table=rebalance_test_table_9 [id=5f28187b5b7a472f8429ed627f138f76]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:33.414637 31246 tablet_service.cc:1467] Processing CreateTablet for tablet c43ff8fc0b4f4f799a6f1bbd677166da (DEFAULT_TABLE table=rebalance_test_table_9 [id=5f28187b5b7a472f8429ed627f138f76]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:33.414894 30982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c43ff8fc0b4f4f799a6f1bbd677166da. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:33.414893 31115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c43ff8fc0b4f4f799a6f1bbd677166da. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:33.414898 31246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c43ff8fc0b4f4f799a6f1bbd677166da. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:33.415513 30843 tablet_service.cc:1467] Processing CreateTablet for tablet 342cec2187904922890444a7e5dd181b (DEFAULT_TABLE table=rebalance_test_table_9 [id=5f28187b5b7a472f8429ed627f138f76]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:33.415762 30843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 342cec2187904922890444a7e5dd181b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:33.416927 31245 tablet_service.cc:1467] Processing CreateTablet for tablet 342cec2187904922890444a7e5dd181b (DEFAULT_TABLE table=rebalance_test_table_9 [id=5f28187b5b7a472f8429ed627f138f76]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:33.417094 32532 tablet_bootstrap.cc:492] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929: Bootstrap starting.
I20240502 14:00:33.417150 31245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 342cec2187904922890444a7e5dd181b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:33.417866 32532 tablet_bootstrap.cc:654] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:33.417982 32533 tablet_bootstrap.cc:492] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f: Bootstrap starting.
I20240502 14:00:33.418084 31244 tablet_service.cc:1467] Processing CreateTablet for tablet 1f3459ebf25f45dd810046420f5dcd81 (DEFAULT_TABLE table=rebalance_test_table_9 [id=5f28187b5b7a472f8429ed627f138f76]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:33.418285 31244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f3459ebf25f45dd810046420f5dcd81. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:33.418543 32531 tablet_bootstrap.cc:492] T c43ff8fc0b4f4f799a6f1bbd677166da P 8c61042437444f5f92c76109dfb229fc: Bootstrap starting.
I20240502 14:00:33.418977 32533 tablet_bootstrap.cc:654] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:33.419391 32531 tablet_bootstrap.cc:654] T c43ff8fc0b4f4f799a6f1bbd677166da P 8c61042437444f5f92c76109dfb229fc: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:33.419498 31115 tablet_service.cc:1467] Processing CreateTablet for tablet 1f3459ebf25f45dd810046420f5dcd81 (DEFAULT_TABLE table=rebalance_test_table_9 [id=5f28187b5b7a472f8429ed627f138f76]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:33.419694 31115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f3459ebf25f45dd810046420f5dcd81. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:33.420465 32531 tablet_bootstrap.cc:492] T c43ff8fc0b4f4f799a6f1bbd677166da P 8c61042437444f5f92c76109dfb229fc: No bootstrap required, opened a new log
I20240502 14:00:33.420578 32531 ts_tablet_manager.cc:1367] T c43ff8fc0b4f4f799a6f1bbd677166da P 8c61042437444f5f92c76109dfb229fc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:33.421100 32531 raft_consensus.cc:348] T c43ff8fc0b4f4f799a6f1bbd677166da P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:33.421259 32531 raft_consensus.cc:374] T c43ff8fc0b4f4f799a6f1bbd677166da P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:33.421317 32531 raft_consensus.cc:729] T c43ff8fc0b4f4f799a6f1bbd677166da P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Initialized, Role: FOLLOWER
I20240502 14:00:33.421468 32531 consensus_queue.cc:260] T c43ff8fc0b4f4f799a6f1bbd677166da P 8c61042437444f5f92c76109dfb229fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:33.421636 30982 tablet_service.cc:1467] Processing CreateTablet for tablet 1f3459ebf25f45dd810046420f5dcd81 (DEFAULT_TABLE table=rebalance_test_table_9 [id=5f28187b5b7a472f8429ed627f138f76]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:33.421867 30982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f3459ebf25f45dd810046420f5dcd81. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:33.422183 32531 ts_tablet_manager.cc:1398] T c43ff8fc0b4f4f799a6f1bbd677166da P 8c61042437444f5f92c76109dfb229fc: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:33.422366 32531 tablet_bootstrap.cc:492] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc: Bootstrap starting.
I20240502 14:00:33.423171 32531 tablet_bootstrap.cc:654] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:33.423296 32535 tablet_bootstrap.cc:492] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052: Bootstrap starting.
I20240502 14:00:33.423764 30981 tablet_service.cc:1467] Processing CreateTablet for tablet 342cec2187904922890444a7e5dd181b (DEFAULT_TABLE table=rebalance_test_table_9 [id=5f28187b5b7a472f8429ed627f138f76]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:33.423982 30981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 342cec2187904922890444a7e5dd181b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:33.424196 32535 tablet_bootstrap.cc:654] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:33.424633 32531 tablet_bootstrap.cc:492] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc: No bootstrap required, opened a new log
I20240502 14:00:33.424733 32531 ts_tablet_manager.cc:1367] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:33.425311 32535 tablet_bootstrap.cc:492] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052: No bootstrap required, opened a new log
I20240502 14:00:33.425328 32531 raft_consensus.cc:348] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:33.425395 32535 ts_tablet_manager.cc:1367] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:33.425441 32531 raft_consensus.cc:374] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:33.425482 32531 raft_consensus.cc:729] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Initialized, Role: FOLLOWER
I20240502 14:00:33.425596 32531 consensus_queue.cc:260] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:33.425832 32531 ts_tablet_manager.cc:1398] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:33.426038 32535 raft_consensus.cc:348] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:33.426141 32535 raft_consensus.cc:374] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:33.426184 32535 raft_consensus.cc:729] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14753c93125941acaad7283edc669052, State: Initialized, Role: FOLLOWER
I20240502 14:00:33.426288 32535 consensus_queue.cc:260] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:33.426637 32535 ts_tablet_manager.cc:1398] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:33.426805 32535 tablet_bootstrap.cc:492] T 1f3459ebf25f45dd810046420f5dcd81 P 14753c93125941acaad7283edc669052: Bootstrap starting.
I20240502 14:00:33.426996 32533 tablet_bootstrap.cc:492] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f: No bootstrap required, opened a new log
I20240502 14:00:33.427023 32532 tablet_bootstrap.cc:492] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929: No bootstrap required, opened a new log
I20240502 14:00:33.427062 32533 ts_tablet_manager.cc:1367] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 14:00:33.427089 32532 ts_tablet_manager.cc:1367] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240502 14:00:33.427354 32533 raft_consensus.cc:348] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:33.427421 32533 raft_consensus.cc:374] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:33.427439 32533 raft_consensus.cc:729] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Initialized, Role: FOLLOWER
I20240502 14:00:33.427536 32533 consensus_queue.cc:260] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:33.427501 32532 raft_consensus.cc:348] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:33.427611 32535 tablet_bootstrap.cc:654] T 1f3459ebf25f45dd810046420f5dcd81 P 14753c93125941acaad7283edc669052: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:33.427619 32532 raft_consensus.cc:374] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:33.427662 32532 raft_consensus.cc:729] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Initialized, Role: FOLLOWER
I20240502 14:00:33.427798 32533 ts_tablet_manager.cc:1398] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:33.427788 32532 consensus_queue.cc:260] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:33.428085 32532 ts_tablet_manager.cc:1398] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:33.428228 32532 tablet_bootstrap.cc:492] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929: Bootstrap starting.
I20240502 14:00:33.429045 32532 tablet_bootstrap.cc:654] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:33.430032 32535 tablet_bootstrap.cc:492] T 1f3459ebf25f45dd810046420f5dcd81 P 14753c93125941acaad7283edc669052: No bootstrap required, opened a new log
I20240502 14:00:33.430106 32535 ts_tablet_manager.cc:1367] T 1f3459ebf25f45dd810046420f5dcd81 P 14753c93125941acaad7283edc669052: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 14:00:33.430507 32535 raft_consensus.cc:348] T 1f3459ebf25f45dd810046420f5dcd81 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:33.430605 32535 raft_consensus.cc:374] T 1f3459ebf25f45dd810046420f5dcd81 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:33.430637 32535 raft_consensus.cc:729] T 1f3459ebf25f45dd810046420f5dcd81 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14753c93125941acaad7283edc669052, State: Initialized, Role: FOLLOWER
I20240502 14:00:33.430656 32532 tablet_bootstrap.cc:492] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929: No bootstrap required, opened a new log
I20240502 14:00:33.430727 32532 ts_tablet_manager.cc:1367] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:33.430792 32535 consensus_queue.cc:260] T 1f3459ebf25f45dd810046420f5dcd81 P 14753c93125941acaad7283edc669052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:33.431046 32535 ts_tablet_manager.cc:1398] T 1f3459ebf25f45dd810046420f5dcd81 P 14753c93125941acaad7283edc669052: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:33.431115 32532 raft_consensus.cc:348] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:33.431187 32535 tablet_bootstrap.cc:492] T c43ff8fc0b4f4f799a6f1bbd677166da P 14753c93125941acaad7283edc669052: Bootstrap starting.
I20240502 14:00:33.431195 32532 raft_consensus.cc:374] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:33.431226 32532 raft_consensus.cc:729] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Initialized, Role: FOLLOWER
I20240502 14:00:33.431326 32532 consensus_queue.cc:260] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:33.431574 32532 ts_tablet_manager.cc:1398] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:33.431718 32532 tablet_bootstrap.cc:492] T 342cec2187904922890444a7e5dd181b P 1fab10dcfb8542fca2b965fed6c67929: Bootstrap starting.
I20240502 14:00:33.431943 32535 tablet_bootstrap.cc:654] T c43ff8fc0b4f4f799a6f1bbd677166da P 14753c93125941acaad7283edc669052: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:33.432791 32532 tablet_bootstrap.cc:654] T 342cec2187904922890444a7e5dd181b P 1fab10dcfb8542fca2b965fed6c67929: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:33.433570 32535 tablet_bootstrap.cc:492] T c43ff8fc0b4f4f799a6f1bbd677166da P 14753c93125941acaad7283edc669052: No bootstrap required, opened a new log
I20240502 14:00:33.433650 32535 ts_tablet_manager.cc:1367] T c43ff8fc0b4f4f799a6f1bbd677166da P 14753c93125941acaad7283edc669052: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:33.434036 32535 raft_consensus.cc:348] T c43ff8fc0b4f4f799a6f1bbd677166da P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:33.434115 32535 raft_consensus.cc:374] T c43ff8fc0b4f4f799a6f1bbd677166da P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:33.434145 32535 raft_consensus.cc:729] T c43ff8fc0b4f4f799a6f1bbd677166da P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14753c93125941acaad7283edc669052, State: Initialized, Role: FOLLOWER
I20240502 14:00:33.434227 32535 consensus_queue.cc:260] T c43ff8fc0b4f4f799a6f1bbd677166da P 14753c93125941acaad7283edc669052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:33.434465 32535 ts_tablet_manager.cc:1398] T c43ff8fc0b4f4f799a6f1bbd677166da P 14753c93125941acaad7283edc669052: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:33.435519 32532 tablet_bootstrap.cc:492] T 342cec2187904922890444a7e5dd181b P 1fab10dcfb8542fca2b965fed6c67929: No bootstrap required, opened a new log
I20240502 14:00:33.435606 32532 ts_tablet_manager.cc:1367] T 342cec2187904922890444a7e5dd181b P 1fab10dcfb8542fca2b965fed6c67929: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 14:00:33.436048 32532 raft_consensus.cc:348] T 342cec2187904922890444a7e5dd181b P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:33.436210 32532 raft_consensus.cc:374] T 342cec2187904922890444a7e5dd181b P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:33.436285 32532 raft_consensus.cc:729] T 342cec2187904922890444a7e5dd181b P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Initialized, Role: FOLLOWER
I20240502 14:00:33.436467 32532 consensus_queue.cc:260] T 342cec2187904922890444a7e5dd181b P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:33.436754 32532 ts_tablet_manager.cc:1398] T 342cec2187904922890444a7e5dd181b P 1fab10dcfb8542fca2b965fed6c67929: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:00:33.579284 32527 raft_consensus.cc:482] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:33.579393 32527 raft_consensus.cc:504] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:33.579777 32527 leader_election.cc:288] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14753c93125941acaad7283edc669052 (127.24.66.68:37029), 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:33.580094 31266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f3459ebf25f45dd810046420f5dcd81" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052" is_pre_election: true
I20240502 14:00:33.580112 31134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f3459ebf25f45dd810046420f5dcd81" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc" is_pre_election: true
I20240502 14:00:33.580206 31266 raft_consensus.cc:2454] T 1f3459ebf25f45dd810046420f5dcd81 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 0.
I20240502 14:00:33.580250 31134 raft_consensus.cc:2454] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 0.
I20240502 14:00:33.580543 30936 leader_election.cc:302] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 1fab10dcfb8542fca2b965fed6c67929; no voters:
I20240502 14:00:33.580704 32527 raft_consensus.cc:2781] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:33.580761 32527 raft_consensus.cc:482] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:33.580809 32527 raft_consensus.cc:3037] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:33.581620 32527 raft_consensus.cc:504] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:33.581888 32527 leader_election.cc:288] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 election: Requested vote from peers 14753c93125941acaad7283edc669052 (127.24.66.68:37029), 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:33.582175 31266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f3459ebf25f45dd810046420f5dcd81" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052"
I20240502 14:00:33.582221 31134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f3459ebf25f45dd810046420f5dcd81" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc"
I20240502 14:00:33.582266 31266 raft_consensus.cc:3037] T 1f3459ebf25f45dd810046420f5dcd81 P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:33.582336 31134 raft_consensus.cc:3037] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:33.582984 31266 raft_consensus.cc:2454] T 1f3459ebf25f45dd810046420f5dcd81 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 1.
I20240502 14:00:33.583076 31134 raft_consensus.cc:2454] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 1.
I20240502 14:00:33.583220 30936 leader_election.cc:302] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 1fab10dcfb8542fca2b965fed6c67929; no voters:
I20240502 14:00:33.583432 32527 raft_consensus.cc:2781] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:33.583508 32527 raft_consensus.cc:686] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Becoming Leader. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Running, Role: LEADER
I20240502 14:00:33.583660 32527 consensus_queue.cc:237] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:33.585201 30714 catalog_manager.cc:5484] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66). New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:33.588040 32527 raft_consensus.cc:482] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:33.588105 32527 raft_consensus.cc:504] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:33.588318 32527 leader_election.cc:288] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14753c93125941acaad7283edc669052 (127.24.66.68:37029), 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:33.588624 31266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c43ff8fc0b4f4f799a6f1bbd677166da" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052" is_pre_election: true
I20240502 14:00:33.588658 31134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c43ff8fc0b4f4f799a6f1bbd677166da" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc" is_pre_election: true
I20240502 14:00:33.588750 31266 raft_consensus.cc:2454] T c43ff8fc0b4f4f799a6f1bbd677166da P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 0.
I20240502 14:00:33.588761 31134 raft_consensus.cc:2454] T c43ff8fc0b4f4f799a6f1bbd677166da P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 0.
I20240502 14:00:33.588979 30936 leader_election.cc:302] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 1fab10dcfb8542fca2b965fed6c67929; no voters:
I20240502 14:00:33.589118 32527 raft_consensus.cc:2781] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:33.589172 32527 raft_consensus.cc:482] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:33.589198 32527 raft_consensus.cc:3037] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:33.589834 32527 raft_consensus.cc:504] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:33.590127 32527 leader_election.cc:288] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 election: Requested vote from peers 14753c93125941acaad7283edc669052 (127.24.66.68:37029), 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:33.590384 31266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c43ff8fc0b4f4f799a6f1bbd677166da" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052"
I20240502 14:00:33.590423 31134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c43ff8fc0b4f4f799a6f1bbd677166da" candidate_uuid: "1fab10dcfb8542fca2b965fed6c67929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c61042437444f5f92c76109dfb229fc"
I20240502 14:00:33.590472 31266 raft_consensus.cc:3037] T c43ff8fc0b4f4f799a6f1bbd677166da P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:33.590503 31134 raft_consensus.cc:3037] T c43ff8fc0b4f4f799a6f1bbd677166da P 8c61042437444f5f92c76109dfb229fc [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:33.591126 31266 raft_consensus.cc:2454] T c43ff8fc0b4f4f799a6f1bbd677166da P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 1.
I20240502 14:00:33.591137 31134 raft_consensus.cc:2454] T c43ff8fc0b4f4f799a6f1bbd677166da P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fab10dcfb8542fca2b965fed6c67929 in term 1.
I20240502 14:00:33.591399 30936 leader_election.cc:302] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 1fab10dcfb8542fca2b965fed6c67929; no voters:
I20240502 14:00:33.591557 32527 raft_consensus.cc:2781] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:33.591630 32527 raft_consensus.cc:686] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Becoming Leader. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Running, Role: LEADER
I20240502 14:00:33.591753 32527 consensus_queue.cc:237] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } }
I20240502 14:00:33.593299 30714 catalog_manager.cc:5484] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66). New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:33.689785 32510 raft_consensus.cc:482] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:00:33.689903 32510 raft_consensus.cc:504] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:33.690285 32510 leader_election.cc:288] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395), 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:33.690488 31002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "342cec2187904922890444a7e5dd181b" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929" is_pre_election: true
I20240502 14:00:33.690606 31002 raft_consensus.cc:2454] T 342cec2187904922890444a7e5dd181b P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 0.
I20240502 14:00:33.690696 31266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "342cec2187904922890444a7e5dd181b" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052" is_pre_election: true
I20240502 14:00:33.690816 31266 raft_consensus.cc:2454] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 0.
I20240502 14:00:33.690879 30797 leader_election.cc:302] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fab10dcfb8542fca2b965fed6c67929, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:33.691026 32510 raft_consensus.cc:2781] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:00:33.691093 32510 raft_consensus.cc:482] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:00:33.691131 32510 raft_consensus.cc:3037] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:33.691869 32510 raft_consensus.cc:504] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:33.692123 32510 leader_election.cc:288] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 election: Requested vote from peers 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395), 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:33.692399 31002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "342cec2187904922890444a7e5dd181b" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
I20240502 14:00:33.692502 31002 raft_consensus.cc:3037] T 342cec2187904922890444a7e5dd181b P 1fab10dcfb8542fca2b965fed6c67929 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:33.692458 31266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "342cec2187904922890444a7e5dd181b" candidate_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14753c93125941acaad7283edc669052"
I20240502 14:00:33.692571 31266 raft_consensus.cc:3037] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:33.693364 31002 raft_consensus.cc:2454] T 342cec2187904922890444a7e5dd181b P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 1.
I20240502 14:00:33.693364 31266 raft_consensus.cc:2454] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad5befacd63742a1b5b69fdacdb8cf8f in term 1.
I20240502 14:00:33.693630 30797 leader_election.cc:302] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fab10dcfb8542fca2b965fed6c67929, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:33.693778 32510 raft_consensus.cc:2781] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:33.693859 32510 raft_consensus.cc:686] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Becoming Leader. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Running, Role: LEADER
I20240502 14:00:33.693982 32510 consensus_queue.cc:237] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } }
I20240502 14:00:33.695533 30714 catalog_manager.cc:5484] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: term changed from 0 to 1, leader changed from <none> to ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65). New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:33.822052 32547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:33.822273 32547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:33.826332 32547 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_9'
I20240502 14:00:33.907328 31134 raft_consensus.cc:1261] T c43ff8fc0b4f4f799a6f1bbd677166da P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:33.907716 31266 raft_consensus.cc:1261] T c43ff8fc0b4f4f799a6f1bbd677166da P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:33.907886 32527 consensus_queue.cc:1035] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:33.908174 32305 consensus_queue.cc:1035] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:33.911320 31267 raft_consensus.cc:1261] T 1f3459ebf25f45dd810046420f5dcd81 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:33.911836 31133 raft_consensus.cc:1261] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:33.913884 31002 raft_consensus.cc:1261] T 342cec2187904922890444a7e5dd181b P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:33.914175 31267 raft_consensus.cc:1261] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:00:33.914269 32436 consensus_queue.cc:1035] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:33.914507 32510 consensus_queue.cc:1035] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:33.917397 32305 consensus_queue.cc:1035] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:00:33.917656 32305 consensus_queue.cc:1035] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 104.278 ms
time per row: 0.0521392 ms
I20240502 14:00:33.974395 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.69:35027
--local_ip_for_outbound_sockets=127.24.66.69
--tserver_master_addrs=127.24.66.126:40727
--webserver_port=40107
--webserver_interface=127.24.66.69
--builtin_ntp_servers=127.24.66.84:35285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
W20240502 14:00:34.074868 32609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:34.075029 32609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:34.075073 32609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:34.077989 32609 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:34.078042 32609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:34.078137 32609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.69
I20240502 14:00:34.081425 32609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:35285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.69:35027
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.24.66.69
--webserver_port=40107
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.24.66.126:40727
--never_fsync=true
--heap_profile_path=/tmp/kudu.32609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.69
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:34.082026 32609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:34.082907 32609 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:34.090687 32614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:34.091114 32615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:34.091828 32609 server_base.cc:1034] running on GCE node
W20240502 14:00:34.092381 32617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:34.092675 32609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:34.093107 32609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:34.094265 32609 hybrid_clock.cc:648] HybridClock initialized: now 1714658434094246 us; error 30 us; skew 500 ppm
I20240502 14:00:34.095985 32609 webserver.cc:460] Webserver started at http://127.24.66.69:40107/ using document root <none> and password file <none>
I20240502 14:00:34.096478 32609 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:34.096551 32609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:34.099614 32609 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:34.101279 32623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:34.102176 32609 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:34.102278 32609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "2716cee98b1b4c4591a40e1f466520e5"
format_stamp: "Formatted at 2024-05-02 14:00:23 on dist-test-slave-5npg"
I20240502 14:00:34.102581 32609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:34.122954 32609 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:34.123421 32609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:34.123562 32609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:34.124011 32609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:34.124794 32609 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:34.124840 32609 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:34.124900 32609 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:34.124927 32609 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:34.132634 32609 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.69:35027
I20240502 14:00:34.132683 32736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.69:35027 every 8 connection(s)
I20240502 14:00:34.133479 32609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 14:00:34.139122 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 32609
I20240502 14:00:34.139552 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.70:38291
--local_ip_for_outbound_sockets=127.24.66.70
--tserver_master_addrs=127.24.66.126:40727
--webserver_port=44363
--webserver_interface=127.24.66.70
--builtin_ntp_servers=127.24.66.84:35285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240502 14:00:34.141675 32737 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40727
I20240502 14:00:34.141860 32737 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:34.142315 32737 heartbeater.cc:511] Master 127.24.66.126:40727 requested a full tablet report, sending...
I20240502 14:00:34.143337 30715 ts_manager.cc:194] Re-registered known tserver with Master: 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69:35027)
I20240502 14:00:34.143855 30715 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.69:33933
W20240502 14:00:34.238279 32740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:34.238447 32740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:34.238503 32740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:34.241380 32740 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:34.241431 32740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:34.241518 32740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.70
I20240502 14:00:34.245193 32740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:35285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.70:38291
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.24.66.70
--webserver_port=44363
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.24.66.126:40727
--never_fsync=true
--heap_profile_path=/tmp/kudu.32740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.70
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:34.245880 32740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:34.246819 32740 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:34.254518 32746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:34.255131 32748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:34.255759 32745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:34.256428 32740 server_base.cc:1034] running on GCE node
I20240502 14:00:34.256742 32740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:34.257174 32740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:34.258322 32740 hybrid_clock.cc:648] HybridClock initialized: now 1714658434258298 us; error 42 us; skew 500 ppm
I20240502 14:00:34.259972 32740 webserver.cc:460] Webserver started at http://127.24.66.70:44363/ using document root <none> and password file <none>
I20240502 14:00:34.260505 32740 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:34.260582 32740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:34.263542 32740 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:34.265187 32754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:34.266058 32740 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:34.266141 32740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "f769c13ece0b40179ba120dbebbaba98"
format_stamp: "Formatted at 2024-05-02 14:00:23 on dist-test-slave-5npg"
I20240502 14:00:34.266486 32740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:34.297626 32740 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:34.298101 32740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:34.298243 32740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:34.298707 32740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:34.299459 32740 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:34.299505 32740 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:34.299556 32740 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:34.299587 32740 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:34.308341 32740 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.70:38291
I20240502 14:00:34.308399 401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.70:38291 every 8 connection(s)
I20240502 14:00:34.309324 32740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 14:00:34.315128 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 32740
I20240502 14:00:34.315608 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.71:35161
--local_ip_for_outbound_sockets=127.24.66.71
--tserver_master_addrs=127.24.66.126:40727
--webserver_port=39219
--webserver_interface=127.24.66.71
--builtin_ntp_servers=127.24.66.84:35285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240502 14:00:34.316916 402 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40727
I20240502 14:00:34.317139 402 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:34.317610 402 heartbeater.cc:511] Master 127.24.66.126:40727 requested a full tablet report, sending...
I20240502 14:00:34.318519 30715 ts_manager.cc:194] Re-registered known tserver with Master: f769c13ece0b40179ba120dbebbaba98 (127.24.66.70:38291)
I20240502 14:00:34.319057 30715 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.70:57181
W20240502 14:00:34.416417 405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:34.416568 405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:34.416621 405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:34.419510 405 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:34.419561 405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:34.419669 405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.71
I20240502 14:00:34.423308 405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:35285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.71:35161
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.24.66.71
--webserver_port=39219
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.24.66.126:40727
--never_fsync=true
--heap_profile_path=/tmp/kudu.405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.71
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:34.423942 405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:34.424891 405 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:34.432395 410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:34.432983 411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:34.433601 413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:34.434302 405 server_base.cc:1034] running on GCE node
I20240502 14:00:34.434638 405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:34.435122 405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:34.436285 405 hybrid_clock.cc:648] HybridClock initialized: now 1714658434436267 us; error 34 us; skew 500 ppm
I20240502 14:00:34.437873 405 webserver.cc:460] Webserver started at http://127.24.66.71:39219/ using document root <none> and password file <none>
I20240502 14:00:34.438334 405 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:34.438405 405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:34.441363 405 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.002s
I20240502 14:00:34.442996 419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:34.443915 405 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:34.444005 405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "3daf9e0837c74a919c89e686597ff56b"
format_stamp: "Formatted at 2024-05-02 14:00:24 on dist-test-slave-5npg"
I20240502 14:00:34.444355 405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:34.459990 405 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:34.460522 405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:34.460669 405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:34.461102 405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:34.461874 405 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:34.461920 405 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:34.461969 405 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:34.461998 405 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:34.470495 405 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.71:35161
I20240502 14:00:34.470551 533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.71:35161 every 8 connection(s)
I20240502 14:00:34.471536 405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 14:00:34.477219 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 405
I20240502 14:00:34.477727 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.72:34475
--local_ip_for_outbound_sockets=127.24.66.72
--tserver_master_addrs=127.24.66.126:40727
--webserver_port=39245
--webserver_interface=127.24.66.72
--builtin_ntp_servers=127.24.66.84:35285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240502 14:00:34.478852 534 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40727
I20240502 14:00:34.479022 534 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:34.479435 534 heartbeater.cc:511] Master 127.24.66.126:40727 requested a full tablet report, sending...
I20240502 14:00:34.480289 30715 ts_manager.cc:194] Re-registered known tserver with Master: 3daf9e0837c74a919c89e686597ff56b (127.24.66.71:35161)
I20240502 14:00:34.481045 30715 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.71:46535
W20240502 14:00:34.579015 537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:34.579172 537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:34.579214 537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:34.582072 537 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:34.582125 537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:34.582187 537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.72
I20240502 14:00:34.585431 537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:35285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.72:34475
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.24.66.72
--webserver_port=39245
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.24.66.126:40727
--never_fsync=true
--heap_profile_path=/tmp/kudu.537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.72
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:34.586088 537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:34.587091 537 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:34.594918 542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:34.595485 545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:34.596081 543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:34.596856 537 server_base.cc:1034] running on GCE node
I20240502 14:00:34.597189 537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:34.597642 537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:34.598786 537 hybrid_clock.cc:648] HybridClock initialized: now 1714658434598780 us; error 23 us; skew 500 ppm
I20240502 14:00:34.600369 537 webserver.cc:460] Webserver started at http://127.24.66.72:39245/ using document root <none> and password file <none>
I20240502 14:00:34.600829 537 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:34.600899 537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:34.603739 537 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 14:00:34.605373 551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:34.606379 537 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:34.606487 537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "2bb442cde32f426bb14791852dd81802"
format_stamp: "Formatted at 2024-05-02 14:00:24 on dist-test-slave-5npg"
I20240502 14:00:34.606930 537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:34.624054 537 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:34.624522 537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:34.624660 537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:34.625099 537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:34.625826 537 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:34.625870 537 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:34.625941 537 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:34.625972 537 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:34.634325 537 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.72:34475
I20240502 14:00:34.634356 665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.72:34475 every 8 connection(s)
I20240502 14:00:34.635357 537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 14:00:34.642985 666 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40727
I20240502 14:00:34.643184 666 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:34.643177 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 537
I20240502 14:00:34.643644 666 heartbeater.cc:511] Master 127.24.66.126:40727 requested a full tablet report, sending...
I20240502 14:00:34.644532 30715 ts_manager.cc:194] Re-registered known tserver with Master: 2bb442cde32f426bb14791852dd81802 (127.24.66.72:34475)
I20240502 14:00:34.645152 30715 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.72:35437
W20240502 14:00:34.769783 670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:34.770020 670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:34.773767 670 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240502 14:00:34.904168 30981 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 14:00:34.904166 31246 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 14:00:34.904343 30982 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 14:00:34.917089 31115 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:00:34.917843 329 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:34.918936 30843 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 14:00:34.920421 32671 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:34.923126 32671 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:34.924235 449 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:34.932504 449 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:34.939715 31115 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:00:34.957692 329 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:34.964493 31246 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 14:00:34.974310 600 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:34.975016 600 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:34.996176 30843 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 14:00:35.105757 31246 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 14:00:35.109720 449 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:35.122406 30843 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 14:00:35.122558 32671 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:35.123432 30982 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 14:00:35.128015 600 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:35.145479 32737 heartbeater.cc:503] Master 127.24.66.126:40727 was elected leader, sending a full tablet report...
I20240502 14:00:35.154733 329 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
W20240502 14:00:35.166471 780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:35.166693 780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20240502 14:00:35.169096 31115 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
W20240502 14:00:35.170449 780 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240502 14:00:35.260066 30982 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 14:00:35.261555 32671 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:35.262570 600 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:35.264212 31115 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:00:35.268806 329 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:35.276121 31246 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 14:00:35.279412 449 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:35.294365 30843 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 14:00:35.321349 402 heartbeater.cc:503] Master 127.24.66.126:40727 was elected leader, sending a full tablet report...
I20240502 14:00:35.349680 31246 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 14:00:35.353871 600 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:35.355888 32671 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:35.360114 30982 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 14:00:35.362433 449 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:35.368788 30843 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 14:00:35.385807 329 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:35.401695 31115 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:00:35.439258 31265 consensus_queue.cc:237] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:35.440917 31002 raft_consensus.cc:1261] T 08588e44069d4044b2d09eb02d252f82 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.440999 31134 raft_consensus.cc:1261] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.441305 909 consensus_queue.cc:1035] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.441737 910 consensus_queue.cc:1035] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.443503 910 raft_consensus.cc:2932] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:35.443872 31134 raft_consensus.cc:2932] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:35.446440 30712 catalog_manager.cc:5484] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index -1 to 4, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:35.451120 31199 consensus_peers.cc:485] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 -> Peer 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69:35027): Couldn't send request to peer 2716cee98b1b4c4591a40e1f466520e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 08588e44069d4044b2d09eb02d252f82. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:35.454768 31001 raft_consensus.cc:2932] T 08588e44069d4044b2d09eb02d252f82 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:35.467419 30863 consensus_queue.cc:237] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:35.468863 31265 raft_consensus.cc:1261] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.469245 31001 raft_consensus.cc:1261] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.469588 915 consensus_queue.cc:1035] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.469745 926 consensus_queue.cc:1035] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 14:00:35.476850 30797 consensus_peers.cc:485] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f -> Peer 3daf9e0837c74a919c89e686597ff56b (127.24.66.71:35161): Couldn't send request to peer 3daf9e0837c74a919c89e686597ff56b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b5b3cf5fbe3d4d999cbf910c8e0ca62e. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:35.479625 915 raft_consensus.cc:2932] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:35.480523 31001 raft_consensus.cc:2932] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
W20240502 14:00:35.482380 902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:35.482606 902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20240502 14:00:35.483153 534 heartbeater.cc:503] Master 127.24.66.126:40727 was elected leader, sending a full tablet report...
I20240502 14:00:35.483639 31265 raft_consensus.cc:2932] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:35.483683 30709 catalog_manager.cc:5484] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
W20240502 14:00:35.488461 902 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240502 14:00:35.492889 31001 consensus_queue.cc:237] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:35.494601 30863 raft_consensus.cc:1261] T e109ef8f5d764df7969642dc6de361a2 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.494943 31134 raft_consensus.cc:1261] T e109ef8f5d764df7969642dc6de361a2 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.495381 941 consensus_queue.cc:1035] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.495540 940 consensus_queue.cc:1035] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.498075 940 raft_consensus.cc:2932] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:35.498381 31134 raft_consensus.cc:2932] T e109ef8f5d764df7969642dc6de361a2 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
W20240502 14:00:35.498540 30935 consensus_peers.cc:485] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 -> Peer f769c13ece0b40179ba120dbebbaba98 (127.24.66.70:38291): Couldn't send request to peer f769c13ece0b40179ba120dbebbaba98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e109ef8f5d764df7969642dc6de361a2. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:35.498696 30863 raft_consensus.cc:2932] T e109ef8f5d764df7969642dc6de361a2 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:35.501560 30712 catalog_manager.cc:5484] T e109ef8f5d764df7969642dc6de361a2 P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:35.510375 30863 consensus_queue.cc:237] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:35.511790 31265 raft_consensus.cc:1261] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.512013 31001 raft_consensus.cc:1261] T 342cec2187904922890444a7e5dd181b P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.512377 915 consensus_queue.cc:1035] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.513139 915 consensus_queue.cc:1035] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.514060 915 raft_consensus.cc:2932] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:35.515082 31265 raft_consensus.cc:2932] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:35.516202 31002 raft_consensus.cc:2932] T 342cec2187904922890444a7e5dd181b P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
W20240502 14:00:35.516678 30794 consensus_peers.cc:485] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f -> Peer 2bb442cde32f426bb14791852dd81802 (127.24.66.72:34475): Couldn't send request to peer 2bb442cde32f426bb14791852dd81802. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 342cec2187904922890444a7e5dd181b. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:35.517158 30714 catalog_manager.cc:5484] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:35.525666 31134 consensus_queue.cc:237] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:35.527032 31002 raft_consensus.cc:1261] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.527349 30863 raft_consensus.cc:1261] T 2e07f0adca594453bad9433ed0017580 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.527379 968 consensus_queue.cc:1035] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.528064 968 consensus_queue.cc:1035] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.529927 968 raft_consensus.cc:2932] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:35.530365 30863 raft_consensus.cc:2932] T 2e07f0adca594453bad9433ed0017580 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:35.530488 31002 raft_consensus.cc:2932] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:35.532560 30709 catalog_manager.cc:5484] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index -1 to 4, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:35.536027 31067 consensus_peers.cc:485] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc -> Peer 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69:35027): Couldn't send request to peer 2716cee98b1b4c4591a40e1f466520e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2e07f0adca594453bad9433ed0017580. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:35.541566 31002 consensus_queue.cc:237] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:35.543035 31134 raft_consensus.cc:1261] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.543341 31265 raft_consensus.cc:1261] T 8ec88a191b8d4188985b5e3c91a2a031 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.543746 941 consensus_queue.cc:1035] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.543932 940 consensus_queue.cc:1035] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.546094 951 raft_consensus.cc:2932] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:35.546450 31134 raft_consensus.cc:2932] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
W20240502 14:00:35.546689 30935 consensus_peers.cc:485] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 -> Peer 3daf9e0837c74a919c89e686597ff56b (127.24.66.71:35161): Couldn't send request to peer 3daf9e0837c74a919c89e686597ff56b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ec88a191b8d4188985b5e3c91a2a031. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:35.548660 31265 raft_consensus.cc:2932] T 8ec88a191b8d4188985b5e3c91a2a031 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:35.549587 30712 catalog_manager.cc:5484] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:35.563426 31002 consensus_queue.cc:237] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:35.565707 30863 raft_consensus.cc:1261] T 42ecb8fd71404d7b9eb25af03d2200af P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:00:35.566021 30935 consensus_peers.cc:485] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 -> Peer f769c13ece0b40179ba120dbebbaba98 (127.24.66.70:38291): Couldn't send request to peer f769c13ece0b40179ba120dbebbaba98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 42ecb8fd71404d7b9eb25af03d2200af. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:35.566220 941 consensus_queue.cc:1035] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.566534 31134 raft_consensus.cc:1261] T 42ecb8fd71404d7b9eb25af03d2200af P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.567893 941 consensus_queue.cc:1035] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.571451 32305 raft_consensus.cc:2932] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:35.572721 31134 raft_consensus.cc:2932] T 42ecb8fd71404d7b9eb25af03d2200af P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:35.574344 30863 raft_consensus.cc:2932] T 42ecb8fd71404d7b9eb25af03d2200af P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:35.576274 30711 catalog_manager.cc:5484] T 42ecb8fd71404d7b9eb25af03d2200af P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:35.586931 30863 consensus_queue.cc:237] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:35.589064 31002 raft_consensus.cc:1261] T 8da28cc4aef043baaa2282f84afd4abf P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:00:35.589151 30794 consensus_peers.cc:485] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f -> Peer 2bb442cde32f426bb14791852dd81802 (127.24.66.72:34475): Couldn't send request to peer 2bb442cde32f426bb14791852dd81802. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da28cc4aef043baaa2282f84afd4abf. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:35.589191 31134 raft_consensus.cc:1261] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.589507 32584 consensus_queue.cc:1035] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.589730 926 consensus_queue.cc:1035] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.591275 32584 raft_consensus.cc:2932] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:35.591745 31002 raft_consensus.cc:2932] T 8da28cc4aef043baaa2282f84afd4abf P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:35.595613 30715 catalog_manager.cc:5484] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:35.597592 31134 raft_consensus.cc:2932] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:35.602553 32671 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:35.605005 600 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:35.608719 30982 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 14:00:35.609195 31246 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 14:00:35.616175 449 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:35.618763 31265 consensus_queue.cc:237] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:35.620152 30863 raft_consensus.cc:1261] T 954fa01ea4d04842b0218b33de347aa1 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.620586 31115 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:00:35.620635 31002 raft_consensus.cc:1261] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:00:35.620718 31199 consensus_peers.cc:485] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 -> Peer 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69:35027): Couldn't send request to peer 2716cee98b1b4c4591a40e1f466520e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 954fa01ea4d04842b0218b33de347aa1. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:35.621006 32525 consensus_queue.cc:1035] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.621379 910 consensus_queue.cc:1035] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.637701 30843 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 14:00:35.638480 329 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:35.646968 666 heartbeater.cc:503] Master 127.24.66.126:40727 was elected leader, sending a full tablet report...
I20240502 14:00:35.663962 32525 raft_consensus.cc:2932] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:35.664999 31002 raft_consensus.cc:2932] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:35.667610 30707 catalog_manager.cc:5484] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index -1 to 4, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:35.669607 30863 raft_consensus.cc:2932] T 954fa01ea4d04842b0218b33de347aa1 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:35.678547 31002 consensus_queue.cc:237] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:35.679939 31265 raft_consensus.cc:1261] T 1f3459ebf25f45dd810046420f5dcd81 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.680419 940 consensus_queue.cc:1035] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 14:00:35.680615 30935 consensus_peers.cc:485] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 -> Peer f769c13ece0b40179ba120dbebbaba98 (127.24.66.70:38291): Couldn't send request to peer f769c13ece0b40179ba120dbebbaba98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1f3459ebf25f45dd810046420f5dcd81. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:35.681669 31134 raft_consensus.cc:1261] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.682034 951 consensus_queue.cc:1035] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.682339 941 raft_consensus.cc:2932] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:35.684005 31265 raft_consensus.cc:2932] T 1f3459ebf25f45dd810046420f5dcd81 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:35.684886 30715 catalog_manager.cc:5484] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:35.687672 31133 raft_consensus.cc:2932] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
Runtime error: ksck discovered errors
I20240502 14:00:35.695518 31002 consensus_queue.cc:237] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:35.696892 31265 raft_consensus.cc:1261] T c43ff8fc0b4f4f799a6f1bbd677166da P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.696971 31133 raft_consensus.cc:1261] T c43ff8fc0b4f4f799a6f1bbd677166da P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:00:35.696981 30935 consensus_peers.cc:485] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 -> Peer 3daf9e0837c74a919c89e686597ff56b (127.24.66.71:35161): Couldn't send request to peer 3daf9e0837c74a919c89e686597ff56b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c43ff8fc0b4f4f799a6f1bbd677166da. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:35.697278 940 consensus_queue.cc:1035] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.697459 941 consensus_queue.cc:1035] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.699087 32305 raft_consensus.cc:2932] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:35.699368 31265 raft_consensus.cc:2932] T c43ff8fc0b4f4f799a6f1bbd677166da P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:35.699503 31133 raft_consensus.cc:2932] T c43ff8fc0b4f4f799a6f1bbd677166da P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:35.701778 30707 catalog_manager.cc:5484] T c43ff8fc0b4f4f799a6f1bbd677166da P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:35.709772 31133 consensus_queue.cc:237] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:35.711167 31265 raft_consensus.cc:1261] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.711560 967 consensus_queue.cc:1035] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.712733 31002 raft_consensus.cc:1261] T cc64ea7db2064251bed364b800b36b87 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.713291 32250 consensus_queue.cc:1035] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.713516 968 raft_consensus.cc:2932] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:35.714002 31265 raft_consensus.cc:2932] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:35.716229 30715 catalog_manager.cc:5484] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:35.717451 31002 raft_consensus.cc:2932] T cc64ea7db2064251bed364b800b36b87 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
W20240502 14:00:35.720227 31067 consensus_peers.cc:485] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc -> Peer 2bb442cde32f426bb14791852dd81802 (127.24.66.72:34475): Couldn't send request to peer 2bb442cde32f426bb14791852dd81802. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc64ea7db2064251bed364b800b36b87. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:35.725158 31002 consensus_queue.cc:237] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:35.726475 30863 raft_consensus.cc:1261] T be25ac1c6e7242519ff636242718cb79 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:00:35.726682 30935 consensus_peers.cc:485] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 -> Peer f769c13ece0b40179ba120dbebbaba98 (127.24.66.70:38291): Couldn't send request to peer f769c13ece0b40179ba120dbebbaba98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: be25ac1c6e7242519ff636242718cb79. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:35.726819 31265 raft_consensus.cc:1261] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.726960 941 consensus_queue.cc:1035] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.727458 941 consensus_queue.cc:1035] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.728783 940 raft_consensus.cc:2932] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:35.729589 31265 raft_consensus.cc:2932] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:35.730660 30863 raft_consensus.cc:2932] T be25ac1c6e7242519ff636242718cb79 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:35.732717 30708 catalog_manager.cc:5484] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:35.741675 30863 consensus_queue.cc:237] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:35.743121 31265 raft_consensus.cc:1261] T aaca0594b68146cdbd18946f123693be P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.743423 31002 raft_consensus.cc:1261] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.743511 933 consensus_queue.cc:1035] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.743830 926 consensus_queue.cc:1035] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.745287 926 raft_consensus.cc:2932] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:35.745680 31265 raft_consensus.cc:2932] T aaca0594b68146cdbd18946f123693be P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:35.748160 30712 catalog_manager.cc:5484] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index -1 to 4, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:35.749536 30794 consensus_peers.cc:485] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f -> Peer 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69:35027): Couldn't send request to peer 2716cee98b1b4c4591a40e1f466520e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: aaca0594b68146cdbd18946f123693be. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:35.751794 31002 raft_consensus.cc:2932] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:35.757833 31002 consensus_queue.cc:237] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:35.759300 31265 raft_consensus.cc:1261] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.759677 941 consensus_queue.cc:1035] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.759831 31133 raft_consensus.cc:1261] T 8c77a03ce6a142debba75e0d387aa8ec P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.761394 941 consensus_queue.cc:1035] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.761763 941 raft_consensus.cc:2932] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
W20240502 14:00:35.763917 30933 consensus_peers.cc:485] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 -> Peer 2bb442cde32f426bb14791852dd81802 (127.24.66.72:34475): Couldn't send request to peer 2bb442cde32f426bb14791852dd81802. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8c77a03ce6a142debba75e0d387aa8ec. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:35.763952 31133 raft_consensus.cc:2932] T 8c77a03ce6a142debba75e0d387aa8ec P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:35.764130 31265 raft_consensus.cc:2932] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:35.765678 30707 catalog_manager.cc:5484] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:35.774611 31265 consensus_queue.cc:237] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:35.776278 30863 raft_consensus.cc:1261] T 46b0a06884ad48da9b060d2810d5c782 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.776502 31002 raft_consensus.cc:1261] T 46b0a06884ad48da9b060d2810d5c782 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.776952 32525 consensus_queue.cc:1035] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.777132 909 consensus_queue.cc:1035] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.779294 910 raft_consensus.cc:2932] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
W20240502 14:00:35.779754 31201 consensus_peers.cc:485] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 -> Peer 3daf9e0837c74a919c89e686597ff56b (127.24.66.71:35161): Couldn't send request to peer 3daf9e0837c74a919c89e686597ff56b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 46b0a06884ad48da9b060d2810d5c782. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:35.779704 31002 raft_consensus.cc:2932] T 46b0a06884ad48da9b060d2810d5c782 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:35.781666 30863 raft_consensus.cc:2932] T 46b0a06884ad48da9b060d2810d5c782 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:35.782426 30712 catalog_manager.cc:5484] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:35.795882 31002 consensus_queue.cc:237] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:35.797531 30863 raft_consensus.cc:1261] T 5702766b82f24e96a4cb5da5f051fc3f P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:00:35.797664 30935 consensus_peers.cc:485] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 -> Peer f769c13ece0b40179ba120dbebbaba98 (127.24.66.70:38291): Couldn't send request to peer f769c13ece0b40179ba120dbebbaba98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5702766b82f24e96a4cb5da5f051fc3f. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:35.797722 31133 raft_consensus.cc:1261] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.797916 32305 consensus_queue.cc:1035] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.798225 940 consensus_queue.cc:1035] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.800194 32305 raft_consensus.cc:2932] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:35.802016 30863 raft_consensus.cc:2932] T 5702766b82f24e96a4cb5da5f051fc3f P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:35.802879 31133 raft_consensus.cc:2932] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:35.803164 30707 catalog_manager.cc:5484] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:35.813799 31133 consensus_queue.cc:237] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:35.815308 30863 raft_consensus.cc:1261] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.815656 31265 raft_consensus.cc:1261] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.816135 967 consensus_queue.cc:1035] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 14:00:35.816295 31067 consensus_peers.cc:485] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc -> Peer 2bb442cde32f426bb14791852dd81802 (127.24.66.72:34475): Couldn't send request to peer 2bb442cde32f426bb14791852dd81802. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 03fb6c61ce4c4ccdafdc9ca7a40db4f5. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:35.816401 968 consensus_queue.cc:1035] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.818075 967 raft_consensus.cc:2932] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:35.820319 30863 raft_consensus.cc:2932] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:35.822049 30708 catalog_manager.cc:5484] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:35.831176 31265 raft_consensus.cc:2932] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:35.839412 30863 consensus_queue.cc:237] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:35.840939 31265 raft_consensus.cc:1261] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.841207 31133 raft_consensus.cc:1261] T d0409950632d4eeead8a826034b5a9e4 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.841307 915 consensus_queue.cc:1035] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.841603 926 consensus_queue.cc:1035] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 14:00:35.841776 30794 consensus_peers.cc:485] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f -> Peer 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69:35027): Couldn't send request to peer 2716cee98b1b4c4591a40e1f466520e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d0409950632d4eeead8a826034b5a9e4. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:35.843390 915 raft_consensus.cc:2932] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:35.843842 31133 raft_consensus.cc:2932] T d0409950632d4eeead8a826034b5a9e4 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:35.846571 30712 catalog_manager.cc:5484] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index -1 to 4, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:35.847632 31265 raft_consensus.cc:2932] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:35.854950 31265 consensus_queue.cc:237] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
W20240502 14:00:35.855862 1058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
I20240502 14:00:35.856381 31002 raft_consensus.cc:1261] T 2051076a81274b688554c218c107ee1c P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:35.856384 30863 raft_consensus.cc:1261] T 2051076a81274b688554c218c107ee1c P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:00:35.856531 1058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20240502 14:00:35.856793 910 consensus_queue.cc:1035] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 14:00:35.857251 31201 consensus_peers.cc:485] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 -> Peer 3daf9e0837c74a919c89e686597ff56b (127.24.66.71:35161): Couldn't send request to peer 3daf9e0837c74a919c89e686597ff56b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2051076a81274b688554c218c107ee1c. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:35.857370 910 consensus_queue.cc:1035] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:35.858547 909 raft_consensus.cc:2932] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:35.859416 31002 raft_consensus.cc:2932] T 2051076a81274b688554c218c107ee1c P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:35.859985 30863 raft_consensus.cc:2932] T 2051076a81274b688554c218c107ee1c P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
W20240502 14:00:35.860833 1058 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240502 14:00:35.861650 30712 catalog_manager.cc:5484] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:35.869459 1134 ts_tablet_manager.cc:902] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 3daf9e0837c74a919c89e686597ff56b: Initiating tablet copy from peer ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
I20240502 14:00:35.870306 1134 tablet_copy_client.cc:308] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.65:39553
I20240502 14:00:35.875497 30887 tablet_copy_service.cc:140] P ad5befacd63742a1b5b69fdacdb8cf8f: Received BeginTabletCopySession request for tablet b5b3cf5fbe3d4d999cbf910c8e0ca62e from peer 3daf9e0837c74a919c89e686597ff56b ({username='slave'} at 127.24.66.71:50625)
I20240502 14:00:35.875643 30887 tablet_copy_service.cc:161] P ad5befacd63742a1b5b69fdacdb8cf8f: Beginning new tablet copy session on tablet b5b3cf5fbe3d4d999cbf910c8e0ca62e from peer 3daf9e0837c74a919c89e686597ff56b at {username='slave'} at 127.24.66.71:50625: session id = 3daf9e0837c74a919c89e686597ff56b-b5b3cf5fbe3d4d999cbf910c8e0ca62e
I20240502 14:00:35.877084 30887 tablet_copy_source_session.cc:205] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:35.880633 1134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5b3cf5fbe3d4d999cbf910c8e0ca62e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.887388 1134 tablet_copy_client.cc:783] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:35.887596 1134 tablet_copy_client.cc:647] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:35.889796 1137 tablet_copy_client.cc:1002] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:35.932530 1220 ts_tablet_manager.cc:902] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b: Initiating tablet copy from peer 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:35.933156 1220 tablet_copy_client.cc:308] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.66:36395
I20240502 14:00:35.938205 31023 tablet_copy_service.cc:140] P 1fab10dcfb8542fca2b965fed6c67929: Received BeginTabletCopySession request for tablet 8ec88a191b8d4188985b5e3c91a2a031 from peer 3daf9e0837c74a919c89e686597ff56b ({username='slave'} at 127.24.66.71:34075)
I20240502 14:00:35.938354 31023 tablet_copy_service.cc:161] P 1fab10dcfb8542fca2b965fed6c67929: Beginning new tablet copy session on tablet 8ec88a191b8d4188985b5e3c91a2a031 from peer 3daf9e0837c74a919c89e686597ff56b at {username='slave'} at 127.24.66.71:34075: session id = 3daf9e0837c74a919c89e686597ff56b-8ec88a191b8d4188985b5e3c91a2a031
I20240502 14:00:35.939823 31023 tablet_copy_source_session.cc:205] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:35.941082 1220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ec88a191b8d4188985b5e3c91a2a031. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.943560 1220 tablet_copy_client.cc:783] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:35.943753 1220 tablet_copy_client.cc:647] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:35.946487 1225 tablet_copy_client.cc:1002] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:35.950579 1235 ts_tablet_manager.cc:902] T e109ef8f5d764df7969642dc6de361a2 P f769c13ece0b40179ba120dbebbaba98: Initiating tablet copy from peer 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:35.951694 1235 tablet_copy_client.cc:308] T e109ef8f5d764df7969642dc6de361a2 P f769c13ece0b40179ba120dbebbaba98: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.66:36395
I20240502 14:00:35.960418 31023 tablet_copy_service.cc:140] P 1fab10dcfb8542fca2b965fed6c67929: Received BeginTabletCopySession request for tablet e109ef8f5d764df7969642dc6de361a2 from peer f769c13ece0b40179ba120dbebbaba98 ({username='slave'} at 127.24.66.70:36955)
I20240502 14:00:35.960526 31023 tablet_copy_service.cc:161] P 1fab10dcfb8542fca2b965fed6c67929: Beginning new tablet copy session on tablet e109ef8f5d764df7969642dc6de361a2 from peer f769c13ece0b40179ba120dbebbaba98 at {username='slave'} at 127.24.66.70:36955: session id = f769c13ece0b40179ba120dbebbaba98-e109ef8f5d764df7969642dc6de361a2
I20240502 14:00:35.961534 31023 tablet_copy_source_session.cc:205] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:35.964823 1235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e109ef8f5d764df7969642dc6de361a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:35.971902 1235 tablet_copy_client.cc:783] T e109ef8f5d764df7969642dc6de361a2 P f769c13ece0b40179ba120dbebbaba98: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:35.972172 1235 tablet_copy_client.cc:647] T e109ef8f5d764df7969642dc6de361a2 P f769c13ece0b40179ba120dbebbaba98: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:35.976112 30982 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 14:00:35.977684 31246 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 14:00:35.981456 449 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:35.988210 329 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:35.989274 30843 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 14:00:35.992271 31115 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:00:36.006932 1272 ts_tablet_manager.cc:902] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5: Initiating tablet copy from peer 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:36.009862 1241 tablet_copy_client.cc:1002] T e109ef8f5d764df7969642dc6de361a2 P f769c13ece0b40179ba120dbebbaba98: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:36.011348 1272 tablet_copy_client.cc:308] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:37029
I20240502 14:00:36.016252 32671 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:36.021847 600 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:36.025488 31288 tablet_copy_service.cc:140] P 14753c93125941acaad7283edc669052: Received BeginTabletCopySession request for tablet 08588e44069d4044b2d09eb02d252f82 from peer 2716cee98b1b4c4591a40e1f466520e5 ({username='slave'} at 127.24.66.69:42815)
I20240502 14:00:36.025645 31288 tablet_copy_service.cc:161] P 14753c93125941acaad7283edc669052: Beginning new tablet copy session on tablet 08588e44069d4044b2d09eb02d252f82 from peer 2716cee98b1b4c4591a40e1f466520e5 at {username='slave'} at 127.24.66.69:42815: session id = 2716cee98b1b4c4591a40e1f466520e5-08588e44069d4044b2d09eb02d252f82
I20240502 14:00:36.027027 31288 tablet_copy_source_session.cc:205] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:36.030472 1284 ts_tablet_manager.cc:902] T 954fa01ea4d04842b0218b33de347aa1 P 2716cee98b1b4c4591a40e1f466520e5: Initiating tablet copy from peer 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:36.030809 1272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08588e44069d4044b2d09eb02d252f82. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.038215 1284 tablet_copy_client.cc:308] T 954fa01ea4d04842b0218b33de347aa1 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:37029
I20240502 14:00:36.038684 31288 tablet_copy_service.cc:140] P 14753c93125941acaad7283edc669052: Received BeginTabletCopySession request for tablet 954fa01ea4d04842b0218b33de347aa1 from peer 2716cee98b1b4c4591a40e1f466520e5 ({username='slave'} at 127.24.66.69:42815)
I20240502 14:00:36.038776 31288 tablet_copy_service.cc:161] P 14753c93125941acaad7283edc669052: Beginning new tablet copy session on tablet 954fa01ea4d04842b0218b33de347aa1 from peer 2716cee98b1b4c4591a40e1f466520e5 at {username='slave'} at 127.24.66.69:42815: session id = 2716cee98b1b4c4591a40e1f466520e5-954fa01ea4d04842b0218b33de347aa1
I20240502 14:00:36.039616 31288 tablet_copy_source_session.cc:205] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:36.040005 1272 tablet_copy_client.cc:783] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:36.040254 1272 tablet_copy_client.cc:647] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:36.040571 1284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 954fa01ea4d04842b0218b33de347aa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.043493 1278 tablet_copy_client.cc:1002] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:36.043576 1284 tablet_copy_client.cc:783] T 954fa01ea4d04842b0218b33de347aa1 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:36.043720 1284 tablet_copy_client.cc:647] T 954fa01ea4d04842b0218b33de347aa1 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:36.044883 1291 tablet_copy_client.cc:1002] T 954fa01ea4d04842b0218b33de347aa1 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Injecting latency into file download: 1500
Runtime error: ksck discovered errors
I20240502 14:00:36.066452 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.73:0
--local_ip_for_outbound_sockets=127.24.66.73
--webserver_interface=127.24.66.73
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40727
--builtin_ntp_servers=127.24.66.84:35285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true
--tablet_copy_download_file_inject_latency_ms=1500
--follower_unavailable_considered_failed_sec=30 with env {}
I20240502 14:00:36.097366 1312 ts_tablet_manager.cc:902] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802: Initiating tablet copy from peer ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
I20240502 14:00:36.098098 1312 tablet_copy_client.cc:308] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.65:39553
I20240502 14:00:36.102232 30887 tablet_copy_service.cc:140] P ad5befacd63742a1b5b69fdacdb8cf8f: Received BeginTabletCopySession request for tablet 342cec2187904922890444a7e5dd181b from peer 2bb442cde32f426bb14791852dd81802 ({username='slave'} at 127.24.66.72:50901)
I20240502 14:00:36.102326 30887 tablet_copy_service.cc:161] P ad5befacd63742a1b5b69fdacdb8cf8f: Beginning new tablet copy session on tablet 342cec2187904922890444a7e5dd181b from peer 2bb442cde32f426bb14791852dd81802 at {username='slave'} at 127.24.66.72:50901: session id = 2bb442cde32f426bb14791852dd81802-342cec2187904922890444a7e5dd181b
I20240502 14:00:36.103024 30887 tablet_copy_source_session.cc:205] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:36.105891 1312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 342cec2187904922890444a7e5dd181b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.110741 1312 tablet_copy_client.cc:783] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:36.110929 1312 tablet_copy_client.cc:647] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:36.111160 1318 ts_tablet_manager.cc:902] T 42ecb8fd71404d7b9eb25af03d2200af P f769c13ece0b40179ba120dbebbaba98: Initiating tablet copy from peer 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:36.111750 1318 tablet_copy_client.cc:308] T 42ecb8fd71404d7b9eb25af03d2200af P f769c13ece0b40179ba120dbebbaba98: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.66:36395
I20240502 14:00:36.112264 31023 tablet_copy_service.cc:140] P 1fab10dcfb8542fca2b965fed6c67929: Received BeginTabletCopySession request for tablet 42ecb8fd71404d7b9eb25af03d2200af from peer f769c13ece0b40179ba120dbebbaba98 ({username='slave'} at 127.24.66.70:36955)
I20240502 14:00:36.112392 31023 tablet_copy_service.cc:161] P 1fab10dcfb8542fca2b965fed6c67929: Beginning new tablet copy session on tablet 42ecb8fd71404d7b9eb25af03d2200af from peer f769c13ece0b40179ba120dbebbaba98 at {username='slave'} at 127.24.66.70:36955: session id = f769c13ece0b40179ba120dbebbaba98-42ecb8fd71404d7b9eb25af03d2200af
I20240502 14:00:36.112434 1316 tablet_copy_client.cc:1002] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:36.113296 31023 tablet_copy_source_session.cc:205] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:36.114826 1318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42ecb8fd71404d7b9eb25af03d2200af. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.116766 1318 tablet_copy_client.cc:783] T 42ecb8fd71404d7b9eb25af03d2200af P f769c13ece0b40179ba120dbebbaba98: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:36.116902 1318 tablet_copy_client.cc:647] T 42ecb8fd71404d7b9eb25af03d2200af P f769c13ece0b40179ba120dbebbaba98: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:36.117961 1319 tablet_copy_client.cc:1002] T 42ecb8fd71404d7b9eb25af03d2200af P f769c13ece0b40179ba120dbebbaba98: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:36.122519 1323 ts_tablet_manager.cc:902] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5: Initiating tablet copy from peer 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:36.123050 1323 tablet_copy_client.cc:308] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.67:34193
I20240502 14:00:36.126987 1328 ts_tablet_manager.cc:902] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98: Initiating tablet copy from peer 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:36.127231 31155 tablet_copy_service.cc:140] P 8c61042437444f5f92c76109dfb229fc: Received BeginTabletCopySession request for tablet 2e07f0adca594453bad9433ed0017580 from peer 2716cee98b1b4c4591a40e1f466520e5 ({username='slave'} at 127.24.66.69:44787)
I20240502 14:00:36.127391 31155 tablet_copy_service.cc:161] P 8c61042437444f5f92c76109dfb229fc: Beginning new tablet copy session on tablet 2e07f0adca594453bad9433ed0017580 from peer 2716cee98b1b4c4591a40e1f466520e5 at {username='slave'} at 127.24.66.69:44787: session id = 2716cee98b1b4c4591a40e1f466520e5-2e07f0adca594453bad9433ed0017580
I20240502 14:00:36.127535 1328 tablet_copy_client.cc:308] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.66:36395
I20240502 14:00:36.128225 31023 tablet_copy_service.cc:140] P 1fab10dcfb8542fca2b965fed6c67929: Received BeginTabletCopySession request for tablet 1f3459ebf25f45dd810046420f5dcd81 from peer f769c13ece0b40179ba120dbebbaba98 ({username='slave'} at 127.24.66.70:36955)
I20240502 14:00:36.128361 31023 tablet_copy_service.cc:161] P 1fab10dcfb8542fca2b965fed6c67929: Beginning new tablet copy session on tablet 1f3459ebf25f45dd810046420f5dcd81 from peer f769c13ece0b40179ba120dbebbaba98 at {username='slave'} at 127.24.66.70:36955: session id = f769c13ece0b40179ba120dbebbaba98-1f3459ebf25f45dd810046420f5dcd81
I20240502 14:00:36.128831 31155 tablet_copy_source_session.cc:205] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:36.129257 31023 tablet_copy_source_session.cc:205] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:36.130077 1323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e07f0adca594453bad9433ed0017580. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.131884 1323 tablet_copy_client.cc:783] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:36.132030 1323 tablet_copy_client.cc:647] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:36.132716 1328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f3459ebf25f45dd810046420f5dcd81. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.133420 1325 tablet_copy_client.cc:1002] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:36.134825 1328 tablet_copy_client.cc:783] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:36.134986 1328 tablet_copy_client.cc:647] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:36.136126 1332 tablet_copy_client.cc:1002] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:36.145254 1333 ts_tablet_manager.cc:902] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802: Initiating tablet copy from peer 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:36.145781 1333 tablet_copy_client.cc:308] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.67:34193
I20240502 14:00:36.149777 31155 tablet_copy_service.cc:140] P 8c61042437444f5f92c76109dfb229fc: Received BeginTabletCopySession request for tablet cc64ea7db2064251bed364b800b36b87 from peer 2bb442cde32f426bb14791852dd81802 ({username='slave'} at 127.24.66.72:50377)
I20240502 14:00:36.149922 31155 tablet_copy_service.cc:161] P 8c61042437444f5f92c76109dfb229fc: Beginning new tablet copy session on tablet cc64ea7db2064251bed364b800b36b87 from peer 2bb442cde32f426bb14791852dd81802 at {username='slave'} at 127.24.66.72:50377: session id = 2bb442cde32f426bb14791852dd81802-cc64ea7db2064251bed364b800b36b87
I20240502 14:00:36.150889 31155 tablet_copy_source_session.cc:205] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:36.151723 1333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc64ea7db2064251bed364b800b36b87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.153939 1333 tablet_copy_client.cc:783] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:36.154099 1333 tablet_copy_client.cc:647] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:36.155072 1337 tablet_copy_client.cc:1002] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:36.166819 1338 ts_tablet_manager.cc:902] T c43ff8fc0b4f4f799a6f1bbd677166da P 3daf9e0837c74a919c89e686597ff56b: Initiating tablet copy from peer 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:36.167346 1338 tablet_copy_client.cc:308] T c43ff8fc0b4f4f799a6f1bbd677166da P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.66:36395
I20240502 14:00:36.167807 31023 tablet_copy_service.cc:140] P 1fab10dcfb8542fca2b965fed6c67929: Received BeginTabletCopySession request for tablet c43ff8fc0b4f4f799a6f1bbd677166da from peer 3daf9e0837c74a919c89e686597ff56b ({username='slave'} at 127.24.66.71:34075)
I20240502 14:00:36.167913 31023 tablet_copy_service.cc:161] P 1fab10dcfb8542fca2b965fed6c67929: Beginning new tablet copy session on tablet c43ff8fc0b4f4f799a6f1bbd677166da from peer 3daf9e0837c74a919c89e686597ff56b at {username='slave'} at 127.24.66.71:34075: session id = 3daf9e0837c74a919c89e686597ff56b-c43ff8fc0b4f4f799a6f1bbd677166da
I20240502 14:00:36.168704 31023 tablet_copy_source_session.cc:205] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:36.169481 1338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c43ff8fc0b4f4f799a6f1bbd677166da. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.171552 1338 tablet_copy_client.cc:783] T c43ff8fc0b4f4f799a6f1bbd677166da P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:36.171707 1338 tablet_copy_client.cc:647] T c43ff8fc0b4f4f799a6f1bbd677166da P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:36.172693 1342 tablet_copy_client.cc:1002] T c43ff8fc0b4f4f799a6f1bbd677166da P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Injecting latency into file download: 1500
W20240502 14:00:36.189316 1311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:36.189528 1311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:36.189603 1311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20240502 14:00:36.190470 1346 ts_tablet_manager.cc:902] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5: Initiating tablet copy from peer ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
I20240502 14:00:36.190994 1346 tablet_copy_client.cc:308] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.65:39553
I20240502 14:00:36.193192 1351 ts_tablet_manager.cc:902] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802: Initiating tablet copy from peer ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
W20240502 14:00:36.193583 1311 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:36.193692 1311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
I20240502 14:00:36.193691 1351 tablet_copy_client.cc:308] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.65:39553
W20240502 14:00:36.193831 1311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.73
I20240502 14:00:36.194105 30887 tablet_copy_service.cc:140] P ad5befacd63742a1b5b69fdacdb8cf8f: Received BeginTabletCopySession request for tablet 8da28cc4aef043baaa2282f84afd4abf from peer 2bb442cde32f426bb14791852dd81802 ({username='slave'} at 127.24.66.72:50901)
I20240502 14:00:36.194255 30887 tablet_copy_service.cc:161] P ad5befacd63742a1b5b69fdacdb8cf8f: Beginning new tablet copy session on tablet 8da28cc4aef043baaa2282f84afd4abf from peer 2bb442cde32f426bb14791852dd81802 at {username='slave'} at 127.24.66.72:50901: session id = 2bb442cde32f426bb14791852dd81802-8da28cc4aef043baaa2282f84afd4abf
I20240502 14:00:36.195106 30887 tablet_copy_source_session.cc:205] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:36.196038 1351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8da28cc4aef043baaa2282f84afd4abf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.196076 30887 tablet_copy_service.cc:140] P ad5befacd63742a1b5b69fdacdb8cf8f: Received BeginTabletCopySession request for tablet aaca0594b68146cdbd18946f123693be from peer 2716cee98b1b4c4591a40e1f466520e5 ({username='slave'} at 127.24.66.69:55949)
I20240502 14:00:36.196144 30887 tablet_copy_service.cc:161] P ad5befacd63742a1b5b69fdacdb8cf8f: Beginning new tablet copy session on tablet aaca0594b68146cdbd18946f123693be from peer 2716cee98b1b4c4591a40e1f466520e5 at {username='slave'} at 127.24.66.69:55949: session id = 2716cee98b1b4c4591a40e1f466520e5-aaca0594b68146cdbd18946f123693be
I20240502 14:00:36.196952 30887 tablet_copy_source_session.cc:205] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:36.198026 1346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aaca0594b68146cdbd18946f123693be. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.198130 1311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:35285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=30
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.73:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--webserver_interface=127.24.66.73
--webserver_port=0
--tablet_copy_download_file_inject_latency_ms=1500
--tserver_master_addrs=127.24.66.126:40727
--never_fsync=true
--heap_profile_path=/tmp/kudu.1311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.73
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:36.198403 1351 tablet_copy_client.cc:783] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:36.198556 1351 tablet_copy_client.cc:647] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:36.199129 1311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:36.200073 1346 tablet_copy_client.cc:783] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:36.200196 1346 tablet_copy_client.cc:647] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:36.200531 1311 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240502 14:00:36.201233 1350 tablet_copy_client.cc:1002] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:36.201443 1353 tablet_copy_client.cc:1002] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802: tablet copy: Injecting latency into file download: 1500
W20240502 14:00:36.209110 1357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:36.209741 1358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:36.210268 1360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:36.211105 1311 server_base.cc:1034] running on GCE node
I20240502 14:00:36.211441 1311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:36.211926 1311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:36.213061 1311 hybrid_clock.cc:648] HybridClock initialized: now 1714658436213036 us; error 40 us; skew 500 ppm
I20240502 14:00:36.214798 1311 webserver.cc:460] Webserver started at http://127.24.66.73:35653/ using document root <none> and password file <none>
I20240502 14:00:36.215301 1311 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:36.215354 1311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:36.215524 1311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:36.217106 1311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/data/instance:
uuid: "06d9e72a952c44b391752da7674fa6c0"
format_stamp: "Formatted at 2024-05-02 14:00:36 on dist-test-slave-5npg"
I20240502 14:00:36.217617 1311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/wal/instance:
uuid: "06d9e72a952c44b391752da7674fa6c0"
format_stamp: "Formatted at 2024-05-02 14:00:36 on dist-test-slave-5npg"
I20240502 14:00:36.218269 1365 ts_tablet_manager.cc:902] T 46b0a06884ad48da9b060d2810d5c782 P 3daf9e0837c74a919c89e686597ff56b: Initiating tablet copy from peer 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:36.218825 1365 tablet_copy_client.cc:308] T 46b0a06884ad48da9b060d2810d5c782 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:37029
I20240502 14:00:36.221313 1311 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 14:00:36.222625 31288 tablet_copy_service.cc:140] P 14753c93125941acaad7283edc669052: Received BeginTabletCopySession request for tablet 46b0a06884ad48da9b060d2810d5c782 from peer 3daf9e0837c74a919c89e686597ff56b ({username='slave'} at 127.24.66.71:44397)
I20240502 14:00:36.222729 31288 tablet_copy_service.cc:161] P 14753c93125941acaad7283edc669052: Beginning new tablet copy session on tablet 46b0a06884ad48da9b060d2810d5c782 from peer 3daf9e0837c74a919c89e686597ff56b at {username='slave'} at 127.24.66.71:44397: session id = 3daf9e0837c74a919c89e686597ff56b-46b0a06884ad48da9b060d2810d5c782
I20240502 14:00:36.223471 1371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:36.223541 31288 tablet_copy_source_session.cc:205] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:36.224377 1365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46b0a06884ad48da9b060d2810d5c782. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.224591 1311 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 14:00:36.224704 1311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/wal
uuid: "06d9e72a952c44b391752da7674fa6c0"
format_stamp: "Formatted at 2024-05-02 14:00:36 on dist-test-slave-5npg"
I20240502 14:00:36.224797 1311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:36.225409 1372 ts_tablet_manager.cc:902] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802: Initiating tablet copy from peer 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:36.225904 1372 tablet_copy_client.cc:308] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.66:36395
I20240502 14:00:36.226608 1365 tablet_copy_client.cc:783] T 46b0a06884ad48da9b060d2810d5c782 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:36.226763 1365 tablet_copy_client.cc:647] T 46b0a06884ad48da9b060d2810d5c782 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:36.228477 1369 tablet_copy_client.cc:1002] T 46b0a06884ad48da9b060d2810d5c782 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:36.229635 31023 tablet_copy_service.cc:140] P 1fab10dcfb8542fca2b965fed6c67929: Received BeginTabletCopySession request for tablet 8c77a03ce6a142debba75e0d387aa8ec from peer 2bb442cde32f426bb14791852dd81802 ({username='slave'} at 127.24.66.72:33919)
I20240502 14:00:36.229734 31023 tablet_copy_service.cc:161] P 1fab10dcfb8542fca2b965fed6c67929: Beginning new tablet copy session on tablet 8c77a03ce6a142debba75e0d387aa8ec from peer 2bb442cde32f426bb14791852dd81802 at {username='slave'} at 127.24.66.72:33919: session id = 2bb442cde32f426bb14791852dd81802-8c77a03ce6a142debba75e0d387aa8ec
I20240502 14:00:36.230683 31023 tablet_copy_source_session.cc:205] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:36.231473 1372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c77a03ce6a142debba75e0d387aa8ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.233229 1372 tablet_copy_client.cc:783] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:36.233373 1372 tablet_copy_client.cc:647] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:36.234426 1374 tablet_copy_client.cc:1002] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:36.237396 1311 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:36.237849 1311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:36.237967 1311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:36.238457 1311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:36.238751 1382 ts_tablet_manager.cc:902] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 2bb442cde32f426bb14791852dd81802: Initiating tablet copy from peer 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:36.239243 1382 tablet_copy_client.cc:308] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 2bb442cde32f426bb14791852dd81802: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.67:34193
I20240502 14:00:36.239491 1311 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:36.239548 1311 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:36.239598 1311 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:36.239627 1311 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:36.239710 31155 tablet_copy_service.cc:140] P 8c61042437444f5f92c76109dfb229fc: Received BeginTabletCopySession request for tablet 03fb6c61ce4c4ccdafdc9ca7a40db4f5 from peer 2bb442cde32f426bb14791852dd81802 ({username='slave'} at 127.24.66.72:50377)
I20240502 14:00:36.239816 31155 tablet_copy_service.cc:161] P 8c61042437444f5f92c76109dfb229fc: Beginning new tablet copy session on tablet 03fb6c61ce4c4ccdafdc9ca7a40db4f5 from peer 2bb442cde32f426bb14791852dd81802 at {username='slave'} at 127.24.66.72:50377: session id = 2bb442cde32f426bb14791852dd81802-03fb6c61ce4c4ccdafdc9ca7a40db4f5
I20240502 14:00:36.240577 31155 tablet_copy_source_session.cc:205] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:36.241320 1382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03fb6c61ce4c4ccdafdc9ca7a40db4f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.243656 1382 tablet_copy_client.cc:783] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 2bb442cde32f426bb14791852dd81802: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:36.243804 1382 tablet_copy_client.cc:647] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 2bb442cde32f426bb14791852dd81802: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:36.244788 1386 tablet_copy_client.cc:1002] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 2bb442cde32f426bb14791852dd81802: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:36.248950 1311 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.73:34725
I20240502 14:00:36.248986 1494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.73:34725 every 8 connection(s)
I20240502 14:00:36.249810 1311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/data/info.pb
I20240502 14:00:36.252851 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 1311
I20240502 14:00:36.253000 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/wal/instance
I20240502 14:00:36.258364 1495 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40727
I20240502 14:00:36.258533 1495 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:36.258955 1495 heartbeater.cc:511] Master 127.24.66.126:40727 requested a full tablet report, sending...
I20240502 14:00:36.259765 30707 ts_manager.cc:194] Registered new tserver with Master: 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73:34725)
I20240502 14:00:36.260251 30707 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.73:44797
I20240502 14:00:36.277730 1508 ts_tablet_manager.cc:902] T 2051076a81274b688554c218c107ee1c P 3daf9e0837c74a919c89e686597ff56b: Initiating tablet copy from peer 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:36.278272 1508 tablet_copy_client.cc:308] T 2051076a81274b688554c218c107ee1c P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:37029
I20240502 14:00:36.278861 31288 tablet_copy_service.cc:140] P 14753c93125941acaad7283edc669052: Received BeginTabletCopySession request for tablet 2051076a81274b688554c218c107ee1c from peer 3daf9e0837c74a919c89e686597ff56b ({username='slave'} at 127.24.66.71:44397)
I20240502 14:00:36.278960 31288 tablet_copy_service.cc:161] P 14753c93125941acaad7283edc669052: Beginning new tablet copy session on tablet 2051076a81274b688554c218c107ee1c from peer 3daf9e0837c74a919c89e686597ff56b at {username='slave'} at 127.24.66.71:44397: session id = 3daf9e0837c74a919c89e686597ff56b-2051076a81274b688554c218c107ee1c
I20240502 14:00:36.279664 31288 tablet_copy_source_session.cc:205] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:36.280455 1508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2051076a81274b688554c218c107ee1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.282496 1508 tablet_copy_client.cc:783] T 2051076a81274b688554c218c107ee1c P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:36.282691 1508 tablet_copy_client.cc:647] T 2051076a81274b688554c218c107ee1c P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:36.283739 1513 tablet_copy_client.cc:1002] T 2051076a81274b688554c218c107ee1c P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:36.292526 1523 ts_tablet_manager.cc:902] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98: Initiating tablet copy from peer 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:36.293131 1523 tablet_copy_client.cc:308] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.66:36395
I20240502 14:00:36.293526 31023 tablet_copy_service.cc:140] P 1fab10dcfb8542fca2b965fed6c67929: Received BeginTabletCopySession request for tablet be25ac1c6e7242519ff636242718cb79 from peer f769c13ece0b40179ba120dbebbaba98 ({username='slave'} at 127.24.66.70:36955)
I20240502 14:00:36.293619 31023 tablet_copy_service.cc:161] P 1fab10dcfb8542fca2b965fed6c67929: Beginning new tablet copy session on tablet be25ac1c6e7242519ff636242718cb79 from peer f769c13ece0b40179ba120dbebbaba98 at {username='slave'} at 127.24.66.70:36955: session id = f769c13ece0b40179ba120dbebbaba98-be25ac1c6e7242519ff636242718cb79
I20240502 14:00:36.294422 31023 tablet_copy_source_session.cc:205] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:36.295253 1523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be25ac1c6e7242519ff636242718cb79. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.297381 1523 tablet_copy_client.cc:783] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:36.297492 1523 tablet_copy_client.cc:647] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:36.298452 1528 tablet_copy_client.cc:1002] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:36.359673 1594 ts_tablet_manager.cc:902] T 5702766b82f24e96a4cb5da5f051fc3f P f769c13ece0b40179ba120dbebbaba98: Initiating tablet copy from peer 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:36.360189 1594 tablet_copy_client.cc:308] T 5702766b82f24e96a4cb5da5f051fc3f P f769c13ece0b40179ba120dbebbaba98: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.66:36395
I20240502 14:00:36.360781 31023 tablet_copy_service.cc:140] P 1fab10dcfb8542fca2b965fed6c67929: Received BeginTabletCopySession request for tablet 5702766b82f24e96a4cb5da5f051fc3f from peer f769c13ece0b40179ba120dbebbaba98 ({username='slave'} at 127.24.66.70:36955)
I20240502 14:00:36.360869 31023 tablet_copy_service.cc:161] P 1fab10dcfb8542fca2b965fed6c67929: Beginning new tablet copy session on tablet 5702766b82f24e96a4cb5da5f051fc3f from peer f769c13ece0b40179ba120dbebbaba98 at {username='slave'} at 127.24.66.70:36955: session id = f769c13ece0b40179ba120dbebbaba98-5702766b82f24e96a4cb5da5f051fc3f
I20240502 14:00:36.361828 31023 tablet_copy_source_session.cc:205] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:36.362898 1594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5702766b82f24e96a4cb5da5f051fc3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.365021 1594 tablet_copy_client.cc:783] T 5702766b82f24e96a4cb5da5f051fc3f P f769c13ece0b40179ba120dbebbaba98: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:36.365183 1594 tablet_copy_client.cc:647] T 5702766b82f24e96a4cb5da5f051fc3f P f769c13ece0b40179ba120dbebbaba98: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:36.367049 1595 tablet_copy_client.cc:1002] T 5702766b82f24e96a4cb5da5f051fc3f P f769c13ece0b40179ba120dbebbaba98: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:36.413936 1614 ts_tablet_manager.cc:902] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5: Initiating tablet copy from peer ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
I20240502 14:00:36.414387 1614 tablet_copy_client.cc:308] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.65:39553
I20240502 14:00:36.414839 30887 tablet_copy_service.cc:140] P ad5befacd63742a1b5b69fdacdb8cf8f: Received BeginTabletCopySession request for tablet d0409950632d4eeead8a826034b5a9e4 from peer 2716cee98b1b4c4591a40e1f466520e5 ({username='slave'} at 127.24.66.69:55949)
I20240502 14:00:36.414932 30887 tablet_copy_service.cc:161] P ad5befacd63742a1b5b69fdacdb8cf8f: Beginning new tablet copy session on tablet d0409950632d4eeead8a826034b5a9e4 from peer 2716cee98b1b4c4591a40e1f466520e5 at {username='slave'} at 127.24.66.69:55949: session id = 2716cee98b1b4c4591a40e1f466520e5-d0409950632d4eeead8a826034b5a9e4
I20240502 14:00:36.415608 30887 tablet_copy_source_session.cc:205] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:36.416405 1614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0409950632d4eeead8a826034b5a9e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:36.418009 1614 tablet_copy_client.cc:783] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:36.418149 1614 tablet_copy_client.cc:647] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:36.419057 1618 tablet_copy_client.cc:1002] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:37.261982 1495 heartbeater.cc:503] Master 127.24.66.126:40727 was elected leader, sending a full tablet report...
I20240502 14:00:37.390174 1134 tablet_copy_client.cc:523] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:37.392082 1134 tablet_bootstrap.cc:492] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 3daf9e0837c74a919c89e686597ff56b: Bootstrap starting.
I20240502 14:00:37.397171 1134 log.cc:824] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 3daf9e0837c74a919c89e686597ff56b: Log is configured to *not* fsync() on all Append() calls
I20240502 14:00:37.416224 1134 tablet_bootstrap.cc:492] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 3daf9e0837c74a919c89e686597ff56b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:37.416512 1134 tablet_bootstrap.cc:492] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 3daf9e0837c74a919c89e686597ff56b: Bootstrap complete.
I20240502 14:00:37.416721 1134 ts_tablet_manager.cc:1367] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 3daf9e0837c74a919c89e686597ff56b: Time spent bootstrapping tablet: real 0.025s user 0.026s sys 0.000s
I20240502 14:00:37.418406 1134 raft_consensus.cc:348] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:37.418628 1134 raft_consensus.cc:729] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3daf9e0837c74a919c89e686597ff56b, State: Initialized, Role: LEARNER
I20240502 14:00:37.418893 1134 consensus_queue.cc:260] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 3daf9e0837c74a919c89e686597ff56b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:37.419608 1134 ts_tablet_manager.cc:1398] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 3daf9e0837c74a919c89e686597ff56b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20240502 14:00:37.420248 30887 tablet_copy_service.cc:342] P ad5befacd63742a1b5b69fdacdb8cf8f: Request end of tablet copy session 3daf9e0837c74a919c89e686597ff56b-b5b3cf5fbe3d4d999cbf910c8e0ca62e received from {username='slave'} at 127.24.66.71:50625
I20240502 14:00:37.420363 30887 tablet_copy_service.cc:433] P ad5befacd63742a1b5b69fdacdb8cf8f: ending tablet copy session 3daf9e0837c74a919c89e686597ff56b-b5b3cf5fbe3d4d999cbf910c8e0ca62e on tablet b5b3cf5fbe3d4d999cbf910c8e0ca62e with peer 3daf9e0837c74a919c89e686597ff56b
I20240502 14:00:37.446882 1220 tablet_copy_client.cc:523] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:37.448033 1220 tablet_bootstrap.cc:492] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b: Bootstrap starting.
I20240502 14:00:37.465278 1220 tablet_bootstrap.cc:492] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:37.465520 1220 tablet_bootstrap.cc:492] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b: Bootstrap complete.
I20240502 14:00:37.465668 1220 ts_tablet_manager.cc:1367] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b: Time spent bootstrapping tablet: real 0.018s user 0.017s sys 0.000s
I20240502 14:00:37.466150 1220 raft_consensus.cc:348] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:37.466244 1220 raft_consensus.cc:729] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3daf9e0837c74a919c89e686597ff56b, State: Initialized, Role: LEARNER
I20240502 14:00:37.466351 1220 consensus_queue.cc:260] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:37.466588 1220 ts_tablet_manager.cc:1398] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:37.467098 31023 tablet_copy_service.cc:342] P 1fab10dcfb8542fca2b965fed6c67929: Request end of tablet copy session 3daf9e0837c74a919c89e686597ff56b-8ec88a191b8d4188985b5e3c91a2a031 received from {username='slave'} at 127.24.66.71:34075
I20240502 14:00:37.467182 31023 tablet_copy_service.cc:433] P 1fab10dcfb8542fca2b965fed6c67929: ending tablet copy session 3daf9e0837c74a919c89e686597ff56b-8ec88a191b8d4188985b5e3c91a2a031 on tablet 8ec88a191b8d4188985b5e3c91a2a031 with peer 3daf9e0837c74a919c89e686597ff56b
I20240502 14:00:37.510320 1235 tablet_copy_client.cc:523] T e109ef8f5d764df7969642dc6de361a2 P f769c13ece0b40179ba120dbebbaba98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:37.512190 1235 tablet_bootstrap.cc:492] T e109ef8f5d764df7969642dc6de361a2 P f769c13ece0b40179ba120dbebbaba98: Bootstrap starting.
I20240502 14:00:37.517971 1235 log.cc:824] T e109ef8f5d764df7969642dc6de361a2 P f769c13ece0b40179ba120dbebbaba98: Log is configured to *not* fsync() on all Append() calls
I20240502 14:00:37.541914 1235 tablet_bootstrap.cc:492] T e109ef8f5d764df7969642dc6de361a2 P f769c13ece0b40179ba120dbebbaba98: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:37.542210 1235 tablet_bootstrap.cc:492] T e109ef8f5d764df7969642dc6de361a2 P f769c13ece0b40179ba120dbebbaba98: Bootstrap complete.
I20240502 14:00:37.542480 1235 ts_tablet_manager.cc:1367] T e109ef8f5d764df7969642dc6de361a2 P f769c13ece0b40179ba120dbebbaba98: Time spent bootstrapping tablet: real 0.030s user 0.029s sys 0.000s
I20240502 14:00:37.543905 1272 tablet_copy_client.cc:523] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:37.545161 1284 tablet_copy_client.cc:523] T 954fa01ea4d04842b0218b33de347aa1 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:37.545287 1235 raft_consensus.cc:348] T e109ef8f5d764df7969642dc6de361a2 P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:37.545644 1235 raft_consensus.cc:729] T e109ef8f5d764df7969642dc6de361a2 P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f769c13ece0b40179ba120dbebbaba98, State: Initialized, Role: LEARNER
I20240502 14:00:37.546046 1272 tablet_bootstrap.cc:492] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap starting.
I20240502 14:00:37.546066 1235 consensus_queue.cc:260] T e109ef8f5d764df7969642dc6de361a2 P f769c13ece0b40179ba120dbebbaba98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:37.546303 1284 tablet_bootstrap.cc:492] T 954fa01ea4d04842b0218b33de347aa1 P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap starting.
I20240502 14:00:37.546808 1235 ts_tablet_manager.cc:1398] T e109ef8f5d764df7969642dc6de361a2 P f769c13ece0b40179ba120dbebbaba98: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20240502 14:00:37.548547 31023 tablet_copy_service.cc:342] P 1fab10dcfb8542fca2b965fed6c67929: Request end of tablet copy session f769c13ece0b40179ba120dbebbaba98-e109ef8f5d764df7969642dc6de361a2 received from {username='slave'} at 127.24.66.70:36955
I20240502 14:00:37.548631 31023 tablet_copy_service.cc:433] P 1fab10dcfb8542fca2b965fed6c67929: ending tablet copy session f769c13ece0b40179ba120dbebbaba98-e109ef8f5d764df7969642dc6de361a2 on tablet e109ef8f5d764df7969642dc6de361a2 with peer f769c13ece0b40179ba120dbebbaba98
I20240502 14:00:37.554212 1272 log.cc:824] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5: Log is configured to *not* fsync() on all Append() calls
I20240502 14:00:37.572285 354 raft_consensus.cc:1203] T e109ef8f5d764df7969642dc6de361a2 P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:37.581903 1284 tablet_bootstrap.cc:492] T 954fa01ea4d04842b0218b33de347aa1 P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:37.582202 1284 tablet_bootstrap.cc:492] T 954fa01ea4d04842b0218b33de347aa1 P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap complete.
I20240502 14:00:37.582475 1284 ts_tablet_manager.cc:1367] T 954fa01ea4d04842b0218b33de347aa1 P 2716cee98b1b4c4591a40e1f466520e5: Time spent bootstrapping tablet: real 0.036s user 0.023s sys 0.006s
I20240502 14:00:37.583514 1272 tablet_bootstrap.cc:492] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:37.583745 1272 tablet_bootstrap.cc:492] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap complete.
I20240502 14:00:37.583891 1272 ts_tablet_manager.cc:1367] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5: Time spent bootstrapping tablet: real 0.038s user 0.017s sys 0.008s
I20240502 14:00:37.584388 1272 raft_consensus.cc:348] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:37.584367 1284 raft_consensus.cc:348] T 954fa01ea4d04842b0218b33de347aa1 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:37.584625 1284 raft_consensus.cc:729] T 954fa01ea4d04842b0218b33de347aa1 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2716cee98b1b4c4591a40e1f466520e5, State: Initialized, Role: LEARNER
I20240502 14:00:37.584625 1272 raft_consensus.cc:729] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2716cee98b1b4c4591a40e1f466520e5, State: Initialized, Role: LEARNER
I20240502 14:00:37.585111 1284 consensus_queue.cc:260] T 954fa01ea4d04842b0218b33de347aa1 P 2716cee98b1b4c4591a40e1f466520e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:37.585110 1272 consensus_queue.cc:260] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:37.585922 1272 ts_tablet_manager.cc:1398] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20240502 14:00:37.585922 1284 ts_tablet_manager.cc:1398] T 954fa01ea4d04842b0218b33de347aa1 P 2716cee98b1b4c4591a40e1f466520e5: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
I20240502 14:00:37.586647 31287 tablet_copy_service.cc:342] P 14753c93125941acaad7283edc669052: Request end of tablet copy session 2716cee98b1b4c4591a40e1f466520e5-08588e44069d4044b2d09eb02d252f82 received from {username='slave'} at 127.24.66.69:42815
I20240502 14:00:37.586747 31287 tablet_copy_service.cc:433] P 14753c93125941acaad7283edc669052: ending tablet copy session 2716cee98b1b4c4591a40e1f466520e5-08588e44069d4044b2d09eb02d252f82 on tablet 08588e44069d4044b2d09eb02d252f82 with peer 2716cee98b1b4c4591a40e1f466520e5
I20240502 14:00:37.587020 31288 tablet_copy_service.cc:342] P 14753c93125941acaad7283edc669052: Request end of tablet copy session 2716cee98b1b4c4591a40e1f466520e5-954fa01ea4d04842b0218b33de347aa1 received from {username='slave'} at 127.24.66.69:42815
I20240502 14:00:37.587087 31288 tablet_copy_service.cc:433] P 14753c93125941acaad7283edc669052: ending tablet copy session 2716cee98b1b4c4591a40e1f466520e5-954fa01ea4d04842b0218b33de347aa1 on tablet 954fa01ea4d04842b0218b33de347aa1 with peer 2716cee98b1b4c4591a40e1f466520e5
I20240502 14:00:37.612819 1312 tablet_copy_client.cc:523] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:37.614846 1312 tablet_bootstrap.cc:492] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802: Bootstrap starting.
I20240502 14:00:37.618266 1318 tablet_copy_client.cc:523] T 42ecb8fd71404d7b9eb25af03d2200af P f769c13ece0b40179ba120dbebbaba98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:37.619689 1318 tablet_bootstrap.cc:492] T 42ecb8fd71404d7b9eb25af03d2200af P f769c13ece0b40179ba120dbebbaba98: Bootstrap starting.
I20240502 14:00:37.625770 1312 log.cc:824] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802: Log is configured to *not* fsync() on all Append() calls
I20240502 14:00:37.633757 1323 tablet_copy_client.cc:523] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:37.635171 1323 tablet_bootstrap.cc:492] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap starting.
I20240502 14:00:37.636541 1328 tablet_copy_client.cc:523] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:37.637730 1328 tablet_bootstrap.cc:492] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98: Bootstrap starting.
I20240502 14:00:37.643465 1318 tablet_bootstrap.cc:492] T 42ecb8fd71404d7b9eb25af03d2200af P f769c13ece0b40179ba120dbebbaba98: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:37.643774 1318 tablet_bootstrap.cc:492] T 42ecb8fd71404d7b9eb25af03d2200af P f769c13ece0b40179ba120dbebbaba98: Bootstrap complete.
I20240502 14:00:37.643958 1318 ts_tablet_manager.cc:1367] T 42ecb8fd71404d7b9eb25af03d2200af P f769c13ece0b40179ba120dbebbaba98: Time spent bootstrapping tablet: real 0.024s user 0.019s sys 0.004s
I20240502 14:00:37.644490 1318 raft_consensus.cc:348] T 42ecb8fd71404d7b9eb25af03d2200af P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:37.644635 1318 raft_consensus.cc:729] T 42ecb8fd71404d7b9eb25af03d2200af P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f769c13ece0b40179ba120dbebbaba98, State: Initialized, Role: LEARNER
I20240502 14:00:37.644793 1318 consensus_queue.cc:260] T 42ecb8fd71404d7b9eb25af03d2200af P f769c13ece0b40179ba120dbebbaba98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:37.645073 1318 ts_tablet_manager.cc:1398] T 42ecb8fd71404d7b9eb25af03d2200af P f769c13ece0b40179ba120dbebbaba98: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:37.646502 31023 tablet_copy_service.cc:342] P 1fab10dcfb8542fca2b965fed6c67929: Request end of tablet copy session f769c13ece0b40179ba120dbebbaba98-42ecb8fd71404d7b9eb25af03d2200af received from {username='slave'} at 127.24.66.70:36955
I20240502 14:00:37.646585 31023 tablet_copy_service.cc:433] P 1fab10dcfb8542fca2b965fed6c67929: ending tablet copy session f769c13ece0b40179ba120dbebbaba98-42ecb8fd71404d7b9eb25af03d2200af on tablet 42ecb8fd71404d7b9eb25af03d2200af with peer f769c13ece0b40179ba120dbebbaba98
I20240502 14:00:37.654237 1312 tablet_bootstrap.cc:492] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:37.654548 1312 tablet_bootstrap.cc:492] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802: Bootstrap complete.
I20240502 14:00:37.654834 1312 ts_tablet_manager.cc:1367] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802: Time spent bootstrapping tablet: real 0.040s user 0.031s sys 0.000s
I20240502 14:00:37.655433 1333 tablet_copy_client.cc:523] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:37.656538 1333 tablet_bootstrap.cc:492] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802: Bootstrap starting.
I20240502 14:00:37.657133 1312 raft_consensus.cc:348] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:37.657382 1312 raft_consensus.cc:729] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2bb442cde32f426bb14791852dd81802, State: Initialized, Role: LEARNER
I20240502 14:00:37.657711 1312 consensus_queue.cc:260] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:37.658495 1312 ts_tablet_manager.cc:1398] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
I20240502 14:00:37.658988 30887 tablet_copy_service.cc:342] P ad5befacd63742a1b5b69fdacdb8cf8f: Request end of tablet copy session 2bb442cde32f426bb14791852dd81802-342cec2187904922890444a7e5dd181b received from {username='slave'} at 127.24.66.72:50901
I20240502 14:00:37.659058 30887 tablet_copy_service.cc:433] P ad5befacd63742a1b5b69fdacdb8cf8f: ending tablet copy session 2bb442cde32f426bb14791852dd81802-342cec2187904922890444a7e5dd181b on tablet 342cec2187904922890444a7e5dd181b with peer 2bb442cde32f426bb14791852dd81802
I20240502 14:00:37.660275 488 raft_consensus.cc:1203] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:37.662127 1323 tablet_bootstrap.cc:492] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:37.662452 1323 tablet_bootstrap.cc:492] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap complete.
I20240502 14:00:37.662652 1323 ts_tablet_manager.cc:1367] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5: Time spent bootstrapping tablet: real 0.028s user 0.024s sys 0.000s
I20240502 14:00:37.663089 1323 raft_consensus.cc:348] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:37.663213 1323 raft_consensus.cc:729] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2716cee98b1b4c4591a40e1f466520e5, State: Initialized, Role: LEARNER
I20240502 14:00:37.663345 1323 consensus_queue.cc:260] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:37.664013 1323 ts_tablet_manager.cc:1398] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:37.664501 31155 tablet_copy_service.cc:342] P 8c61042437444f5f92c76109dfb229fc: Request end of tablet copy session 2716cee98b1b4c4591a40e1f466520e5-2e07f0adca594453bad9433ed0017580 received from {username='slave'} at 127.24.66.69:44787
I20240502 14:00:37.664592 31155 tablet_copy_service.cc:433] P 8c61042437444f5f92c76109dfb229fc: ending tablet copy session 2716cee98b1b4c4591a40e1f466520e5-2e07f0adca594453bad9433ed0017580 on tablet 2e07f0adca594453bad9433ed0017580 with peer 2716cee98b1b4c4591a40e1f466520e5
I20240502 14:00:37.671128 1328 tablet_bootstrap.cc:492] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:37.671358 1328 tablet_bootstrap.cc:492] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98: Bootstrap complete.
I20240502 14:00:37.671512 1328 ts_tablet_manager.cc:1367] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98: Time spent bootstrapping tablet: real 0.034s user 0.019s sys 0.000s
I20240502 14:00:37.671849 1328 raft_consensus.cc:348] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:37.671937 1328 raft_consensus.cc:729] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f769c13ece0b40179ba120dbebbaba98, State: Initialized, Role: LEARNER
I20240502 14:00:37.672029 1328 consensus_queue.cc:260] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:37.672755 1328 ts_tablet_manager.cc:1398] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 14:00:37.672951 1338 tablet_copy_client.cc:523] T c43ff8fc0b4f4f799a6f1bbd677166da P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:37.674011 1338 tablet_bootstrap.cc:492] T c43ff8fc0b4f4f799a6f1bbd677166da P 3daf9e0837c74a919c89e686597ff56b: Bootstrap starting.
I20240502 14:00:37.676518 31023 tablet_copy_service.cc:342] P 1fab10dcfb8542fca2b965fed6c67929: Request end of tablet copy session f769c13ece0b40179ba120dbebbaba98-1f3459ebf25f45dd810046420f5dcd81 received from {username='slave'} at 127.24.66.70:36955
I20240502 14:00:37.676599 31023 tablet_copy_service.cc:433] P 1fab10dcfb8542fca2b965fed6c67929: ending tablet copy session f769c13ece0b40179ba120dbebbaba98-1f3459ebf25f45dd810046420f5dcd81 on tablet 1f3459ebf25f45dd810046420f5dcd81 with peer f769c13ece0b40179ba120dbebbaba98
I20240502 14:00:37.686662 1333 tablet_bootstrap.cc:492] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:37.686882 1333 tablet_bootstrap.cc:492] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802: Bootstrap complete.
I20240502 14:00:37.687017 1333 ts_tablet_manager.cc:1367] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802: Time spent bootstrapping tablet: real 0.031s user 0.017s sys 0.000s
I20240502 14:00:37.687400 1333 raft_consensus.cc:348] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:37.687487 1333 raft_consensus.cc:729] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2bb442cde32f426bb14791852dd81802, State: Initialized, Role: LEARNER
I20240502 14:00:37.687580 1333 consensus_queue.cc:260] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:37.687816 1333 ts_tablet_manager.cc:1398] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 14:00:37.688217 31155 tablet_copy_service.cc:342] P 8c61042437444f5f92c76109dfb229fc: Request end of tablet copy session 2bb442cde32f426bb14791852dd81802-cc64ea7db2064251bed364b800b36b87 received from {username='slave'} at 127.24.66.72:50377
I20240502 14:00:37.688297 31155 tablet_copy_service.cc:433] P 8c61042437444f5f92c76109dfb229fc: ending tablet copy session 2bb442cde32f426bb14791852dd81802-cc64ea7db2064251bed364b800b36b87 on tablet cc64ea7db2064251bed364b800b36b87 with peer 2bb442cde32f426bb14791852dd81802
I20240502 14:00:37.692492 1338 tablet_bootstrap.cc:492] T c43ff8fc0b4f4f799a6f1bbd677166da P 3daf9e0837c74a919c89e686597ff56b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:37.692696 1338 tablet_bootstrap.cc:492] T c43ff8fc0b4f4f799a6f1bbd677166da P 3daf9e0837c74a919c89e686597ff56b: Bootstrap complete.
I20240502 14:00:37.692831 1338 ts_tablet_manager.cc:1367] T c43ff8fc0b4f4f799a6f1bbd677166da P 3daf9e0837c74a919c89e686597ff56b: Time spent bootstrapping tablet: real 0.019s user 0.018s sys 0.000s
I20240502 14:00:37.693142 1338 raft_consensus.cc:348] T c43ff8fc0b4f4f799a6f1bbd677166da P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:37.693230 1338 raft_consensus.cc:729] T c43ff8fc0b4f4f799a6f1bbd677166da P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3daf9e0837c74a919c89e686597ff56b, State: Initialized, Role: LEARNER
I20240502 14:00:37.693321 1338 consensus_queue.cc:260] T c43ff8fc0b4f4f799a6f1bbd677166da P 3daf9e0837c74a919c89e686597ff56b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:37.693548 1338 ts_tablet_manager.cc:1398] T c43ff8fc0b4f4f799a6f1bbd677166da P 3daf9e0837c74a919c89e686597ff56b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:37.693961 31023 tablet_copy_service.cc:342] P 1fab10dcfb8542fca2b965fed6c67929: Request end of tablet copy session 3daf9e0837c74a919c89e686597ff56b-c43ff8fc0b4f4f799a6f1bbd677166da received from {username='slave'} at 127.24.66.71:34075
I20240502 14:00:37.694037 31023 tablet_copy_service.cc:433] P 1fab10dcfb8542fca2b965fed6c67929: ending tablet copy session 3daf9e0837c74a919c89e686597ff56b-c43ff8fc0b4f4f799a6f1bbd677166da on tablet c43ff8fc0b4f4f799a6f1bbd677166da with peer 3daf9e0837c74a919c89e686597ff56b
I20240502 14:00:37.701539 1346 tablet_copy_client.cc:523] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:37.701669 1351 tablet_copy_client.cc:523] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:37.702682 1351 tablet_bootstrap.cc:492] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802: Bootstrap starting.
I20240502 14:00:37.702697 1346 tablet_bootstrap.cc:492] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap starting.
I20240502 14:00:37.717520 354 raft_consensus.cc:1203] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:37.721225 1351 tablet_bootstrap.cc:492] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:37.721454 1351 tablet_bootstrap.cc:492] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802: Bootstrap complete.
I20240502 14:00:37.721597 1351 ts_tablet_manager.cc:1367] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802: Time spent bootstrapping tablet: real 0.019s user 0.019s sys 0.000s
I20240502 14:00:37.721920 1351 raft_consensus.cc:348] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:37.722013 1351 raft_consensus.cc:729] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2bb442cde32f426bb14791852dd81802, State: Initialized, Role: LEARNER
I20240502 14:00:37.722100 1346 tablet_bootstrap.cc:492] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:37.722118 1351 consensus_queue.cc:260] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:37.722337 1346 tablet_bootstrap.cc:492] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap complete.
I20240502 14:00:37.722365 1351 ts_tablet_manager.cc:1398] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:37.722502 1346 ts_tablet_manager.cc:1367] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5: Time spent bootstrapping tablet: real 0.020s user 0.014s sys 0.005s
I20240502 14:00:37.722798 30887 tablet_copy_service.cc:342] P ad5befacd63742a1b5b69fdacdb8cf8f: Request end of tablet copy session 2bb442cde32f426bb14791852dd81802-8da28cc4aef043baaa2282f84afd4abf received from {username='slave'} at 127.24.66.72:50901
I20240502 14:00:37.722875 30887 tablet_copy_service.cc:433] P ad5befacd63742a1b5b69fdacdb8cf8f: ending tablet copy session 2bb442cde32f426bb14791852dd81802-8da28cc4aef043baaa2282f84afd4abf on tablet 8da28cc4aef043baaa2282f84afd4abf with peer 2bb442cde32f426bb14791852dd81802
I20240502 14:00:37.722896 1346 raft_consensus.cc:348] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:37.722985 1346 raft_consensus.cc:729] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2716cee98b1b4c4591a40e1f466520e5, State: Initialized, Role: LEARNER
I20240502 14:00:37.723048 1346 consensus_queue.cc:260] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:37.723346 1346 ts_tablet_manager.cc:1398] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 14:00:37.723886 30887 tablet_copy_service.cc:342] P ad5befacd63742a1b5b69fdacdb8cf8f: Request end of tablet copy session 2716cee98b1b4c4591a40e1f466520e5-aaca0594b68146cdbd18946f123693be received from {username='slave'} at 127.24.66.69:55949
I20240502 14:00:37.723963 30887 tablet_copy_service.cc:433] P ad5befacd63742a1b5b69fdacdb8cf8f: ending tablet copy session 2716cee98b1b4c4591a40e1f466520e5-aaca0594b68146cdbd18946f123693be on tablet aaca0594b68146cdbd18946f123693be with peer 2716cee98b1b4c4591a40e1f466520e5
I20240502 14:00:37.728710 1365 tablet_copy_client.cc:523] T 46b0a06884ad48da9b060d2810d5c782 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:37.729770 1365 tablet_bootstrap.cc:492] T 46b0a06884ad48da9b060d2810d5c782 P 3daf9e0837c74a919c89e686597ff56b: Bootstrap starting.
I20240502 14:00:37.734772 1372 tablet_copy_client.cc:523] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:37.736006 1372 tablet_bootstrap.cc:492] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802: Bootstrap starting.
I20240502 14:00:37.745041 1382 tablet_copy_client.cc:523] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 2bb442cde32f426bb14791852dd81802: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:37.745965 1382 tablet_bootstrap.cc:492] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 2bb442cde32f426bb14791852dd81802: Bootstrap starting.
I20240502 14:00:37.752914 1365 tablet_bootstrap.cc:492] T 46b0a06884ad48da9b060d2810d5c782 P 3daf9e0837c74a919c89e686597ff56b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:37.753176 1365 tablet_bootstrap.cc:492] T 46b0a06884ad48da9b060d2810d5c782 P 3daf9e0837c74a919c89e686597ff56b: Bootstrap complete.
I20240502 14:00:37.753371 1365 ts_tablet_manager.cc:1367] T 46b0a06884ad48da9b060d2810d5c782 P 3daf9e0837c74a919c89e686597ff56b: Time spent bootstrapping tablet: real 0.024s user 0.021s sys 0.003s
I20240502 14:00:37.753769 1365 raft_consensus.cc:348] T 46b0a06884ad48da9b060d2810d5c782 P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:37.753942 1365 raft_consensus.cc:729] T 46b0a06884ad48da9b060d2810d5c782 P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3daf9e0837c74a919c89e686597ff56b, State: Initialized, Role: LEARNER
I20240502 14:00:37.754060 1365 consensus_queue.cc:260] T 46b0a06884ad48da9b060d2810d5c782 P 3daf9e0837c74a919c89e686597ff56b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:37.754415 1365 ts_tablet_manager.cc:1398] T 46b0a06884ad48da9b060d2810d5c782 P 3daf9e0837c74a919c89e686597ff56b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:37.754837 31288 tablet_copy_service.cc:342] P 14753c93125941acaad7283edc669052: Request end of tablet copy session 3daf9e0837c74a919c89e686597ff56b-46b0a06884ad48da9b060d2810d5c782 received from {username='slave'} at 127.24.66.71:44397
I20240502 14:00:37.754913 31288 tablet_copy_service.cc:433] P 14753c93125941acaad7283edc669052: ending tablet copy session 3daf9e0837c74a919c89e686597ff56b-46b0a06884ad48da9b060d2810d5c782 on tablet 46b0a06884ad48da9b060d2810d5c782 with peer 3daf9e0837c74a919c89e686597ff56b
I20240502 14:00:37.760102 1372 tablet_bootstrap.cc:492] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:37.760357 1372 tablet_bootstrap.cc:492] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802: Bootstrap complete.
I20240502 14:00:37.760514 1372 ts_tablet_manager.cc:1367] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802: Time spent bootstrapping tablet: real 0.025s user 0.020s sys 0.003s
I20240502 14:00:37.760849 1372 raft_consensus.cc:348] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:37.760941 1372 raft_consensus.cc:729] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2bb442cde32f426bb14791852dd81802, State: Initialized, Role: LEARNER
I20240502 14:00:37.761031 1372 consensus_queue.cc:260] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:37.761255 1372 ts_tablet_manager.cc:1398] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:37.761866 31023 tablet_copy_service.cc:342] P 1fab10dcfb8542fca2b965fed6c67929: Request end of tablet copy session 2bb442cde32f426bb14791852dd81802-8c77a03ce6a142debba75e0d387aa8ec received from {username='slave'} at 127.24.66.72:33919
I20240502 14:00:37.761941 31023 tablet_copy_service.cc:433] P 1fab10dcfb8542fca2b965fed6c67929: ending tablet copy session 2bb442cde32f426bb14791852dd81802-8c77a03ce6a142debba75e0d387aa8ec on tablet 8c77a03ce6a142debba75e0d387aa8ec with peer 2bb442cde32f426bb14791852dd81802
I20240502 14:00:37.765053 1382 tablet_bootstrap.cc:492] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 2bb442cde32f426bb14791852dd81802: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:37.765240 1382 tablet_bootstrap.cc:492] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 2bb442cde32f426bb14791852dd81802: Bootstrap complete.
I20240502 14:00:37.765348 1382 ts_tablet_manager.cc:1367] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 2bb442cde32f426bb14791852dd81802: Time spent bootstrapping tablet: real 0.019s user 0.019s sys 0.000s
I20240502 14:00:37.765602 1382 raft_consensus.cc:348] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:37.765686 1382 raft_consensus.cc:729] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2bb442cde32f426bb14791852dd81802, State: Initialized, Role: LEARNER
I20240502 14:00:37.765753 1382 consensus_queue.cc:260] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 2bb442cde32f426bb14791852dd81802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:37.765959 1382 ts_tablet_manager.cc:1398] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 2bb442cde32f426bb14791852dd81802: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:37.766444 31155 tablet_copy_service.cc:342] P 8c61042437444f5f92c76109dfb229fc: Request end of tablet copy session 2bb442cde32f426bb14791852dd81802-03fb6c61ce4c4ccdafdc9ca7a40db4f5 received from {username='slave'} at 127.24.66.72:50377
I20240502 14:00:37.766527 31155 tablet_copy_service.cc:433] P 8c61042437444f5f92c76109dfb229fc: ending tablet copy session 2bb442cde32f426bb14791852dd81802-03fb6c61ce4c4ccdafdc9ca7a40db4f5 on tablet 03fb6c61ce4c4ccdafdc9ca7a40db4f5 with peer 2bb442cde32f426bb14791852dd81802
I20240502 14:00:37.766916 620 raft_consensus.cc:1203] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:37.784154 1508 tablet_copy_client.cc:523] T 2051076a81274b688554c218c107ee1c P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:37.785367 1508 tablet_bootstrap.cc:492] T 2051076a81274b688554c218c107ee1c P 3daf9e0837c74a919c89e686597ff56b: Bootstrap starting.
I20240502 14:00:37.798734 1523 tablet_copy_client.cc:523] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:37.800108 1523 tablet_bootstrap.cc:492] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98: Bootstrap starting.
I20240502 14:00:37.809731 1508 tablet_bootstrap.cc:492] T 2051076a81274b688554c218c107ee1c P 3daf9e0837c74a919c89e686597ff56b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:37.810019 1508 tablet_bootstrap.cc:492] T 2051076a81274b688554c218c107ee1c P 3daf9e0837c74a919c89e686597ff56b: Bootstrap complete.
I20240502 14:00:37.810216 1508 ts_tablet_manager.cc:1367] T 2051076a81274b688554c218c107ee1c P 3daf9e0837c74a919c89e686597ff56b: Time spent bootstrapping tablet: real 0.025s user 0.025s sys 0.000s
I20240502 14:00:37.810606 1508 raft_consensus.cc:348] T 2051076a81274b688554c218c107ee1c P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:37.810739 1508 raft_consensus.cc:729] T 2051076a81274b688554c218c107ee1c P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3daf9e0837c74a919c89e686597ff56b, State: Initialized, Role: LEARNER
I20240502 14:00:37.810864 1508 consensus_queue.cc:260] T 2051076a81274b688554c218c107ee1c P 3daf9e0837c74a919c89e686597ff56b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:37.811143 1508 ts_tablet_manager.cc:1398] T 2051076a81274b688554c218c107ee1c P 3daf9e0837c74a919c89e686597ff56b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:37.811723 31288 tablet_copy_service.cc:342] P 14753c93125941acaad7283edc669052: Request end of tablet copy session 3daf9e0837c74a919c89e686597ff56b-2051076a81274b688554c218c107ee1c received from {username='slave'} at 127.24.66.71:44397
I20240502 14:00:37.811810 31288 tablet_copy_service.cc:433] P 14753c93125941acaad7283edc669052: ending tablet copy session 3daf9e0837c74a919c89e686597ff56b-2051076a81274b688554c218c107ee1c on tablet 2051076a81274b688554c218c107ee1c with peer 3daf9e0837c74a919c89e686597ff56b
I20240502 14:00:37.820844 1523 tablet_bootstrap.cc:492] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:37.821070 1523 tablet_bootstrap.cc:492] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98: Bootstrap complete.
I20240502 14:00:37.821206 1523 ts_tablet_manager.cc:1367] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98: Time spent bootstrapping tablet: real 0.021s user 0.018s sys 0.004s
I20240502 14:00:37.821568 1523 raft_consensus.cc:348] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:37.821684 1523 raft_consensus.cc:729] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f769c13ece0b40179ba120dbebbaba98, State: Initialized, Role: LEARNER
I20240502 14:00:37.821822 1523 consensus_queue.cc:260] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:37.822065 1523 ts_tablet_manager.cc:1398] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:37.822769 31023 tablet_copy_service.cc:342] P 1fab10dcfb8542fca2b965fed6c67929: Request end of tablet copy session f769c13ece0b40179ba120dbebbaba98-be25ac1c6e7242519ff636242718cb79 received from {username='slave'} at 127.24.66.70:36955
I20240502 14:00:37.822855 31023 tablet_copy_service.cc:433] P 1fab10dcfb8542fca2b965fed6c67929: ending tablet copy session f769c13ece0b40179ba120dbebbaba98-be25ac1c6e7242519ff636242718cb79 on tablet be25ac1c6e7242519ff636242718cb79 with peer f769c13ece0b40179ba120dbebbaba98
I20240502 14:00:37.827153 354 raft_consensus.cc:1203] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:37.841882 620 raft_consensus.cc:1203] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:37.867352 1594 tablet_copy_client.cc:523] T 5702766b82f24e96a4cb5da5f051fc3f P f769c13ece0b40179ba120dbebbaba98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:37.868829 1594 tablet_bootstrap.cc:492] T 5702766b82f24e96a4cb5da5f051fc3f P f769c13ece0b40179ba120dbebbaba98: Bootstrap starting.
I20240502 14:00:37.890228 1594 tablet_bootstrap.cc:492] T 5702766b82f24e96a4cb5da5f051fc3f P f769c13ece0b40179ba120dbebbaba98: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:37.890478 1594 tablet_bootstrap.cc:492] T 5702766b82f24e96a4cb5da5f051fc3f P f769c13ece0b40179ba120dbebbaba98: Bootstrap complete.
I20240502 14:00:37.890650 1594 ts_tablet_manager.cc:1367] T 5702766b82f24e96a4cb5da5f051fc3f P f769c13ece0b40179ba120dbebbaba98: Time spent bootstrapping tablet: real 0.022s user 0.017s sys 0.003s
I20240502 14:00:37.891083 1594 raft_consensus.cc:348] T 5702766b82f24e96a4cb5da5f051fc3f P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:37.891201 1594 raft_consensus.cc:729] T 5702766b82f24e96a4cb5da5f051fc3f P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f769c13ece0b40179ba120dbebbaba98, State: Initialized, Role: LEARNER
I20240502 14:00:37.891312 1594 consensus_queue.cc:260] T 5702766b82f24e96a4cb5da5f051fc3f P f769c13ece0b40179ba120dbebbaba98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:37.891821 488 raft_consensus.cc:1203] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:37.892872 1594 ts_tablet_manager.cc:1398] T 5702766b82f24e96a4cb5da5f051fc3f P f769c13ece0b40179ba120dbebbaba98: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240502 14:00:37.893313 31023 tablet_copy_service.cc:342] P 1fab10dcfb8542fca2b965fed6c67929: Request end of tablet copy session f769c13ece0b40179ba120dbebbaba98-5702766b82f24e96a4cb5da5f051fc3f received from {username='slave'} at 127.24.66.70:36955
I20240502 14:00:37.893426 31023 tablet_copy_service.cc:433] P 1fab10dcfb8542fca2b965fed6c67929: ending tablet copy session f769c13ece0b40179ba120dbebbaba98-5702766b82f24e96a4cb5da5f051fc3f on tablet 5702766b82f24e96a4cb5da5f051fc3f with peer f769c13ece0b40179ba120dbebbaba98
I20240502 14:00:37.898300 488 raft_consensus.cc:1203] T c43ff8fc0b4f4f799a6f1bbd677166da P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:37.919324 1614 tablet_copy_client.cc:523] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:37.920758 1614 tablet_bootstrap.cc:492] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap starting.
I20240502 14:00:37.924086 32691 raft_consensus.cc:1203] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:37.940701 1614 tablet_bootstrap.cc:492] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:37.940919 1614 tablet_bootstrap.cc:492] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap complete.
I20240502 14:00:37.941059 1614 ts_tablet_manager.cc:1367] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5: Time spent bootstrapping tablet: real 0.020s user 0.016s sys 0.003s
I20240502 14:00:37.941375 1614 raft_consensus.cc:348] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:37.941473 1614 raft_consensus.cc:729] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2716cee98b1b4c4591a40e1f466520e5, State: Initialized, Role: LEARNER
I20240502 14:00:37.941581 1614 consensus_queue.cc:260] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:37.941817 1614 ts_tablet_manager.cc:1398] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:37.942229 30887 tablet_copy_service.cc:342] P ad5befacd63742a1b5b69fdacdb8cf8f: Request end of tablet copy session 2716cee98b1b4c4591a40e1f466520e5-d0409950632d4eeead8a826034b5a9e4 received from {username='slave'} at 127.24.66.69:55949
I20240502 14:00:37.942301 30887 tablet_copy_service.cc:433] P ad5befacd63742a1b5b69fdacdb8cf8f: ending tablet copy session 2716cee98b1b4c4591a40e1f466520e5-d0409950632d4eeead8a826034b5a9e4 on tablet d0409950632d4eeead8a826034b5a9e4 with peer 2716cee98b1b4c4591a40e1f466520e5
I20240502 14:00:37.945003 32691 raft_consensus.cc:1203] T 954fa01ea4d04842b0218b33de347aa1 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:37.952428 32691 raft_consensus.cc:1203] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:37.971469 32691 raft_consensus.cc:1203] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:37.985309 354 raft_consensus.cc:1203] T 42ecb8fd71404d7b9eb25af03d2200af P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:38.008931 488 raft_consensus.cc:1203] T 46b0a06884ad48da9b060d2810d5c782 P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:38.042558 2149 raft_consensus.cc:1050] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929: attempting to promote NON_VOTER f769c13ece0b40179ba120dbebbaba98 to VOTER
I20240502 14:00:38.042934 2149 consensus_queue.cc:237] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } }
I20240502 14:00:38.044108 31134 raft_consensus.cc:1261] T e109ef8f5d764df7969642dc6de361a2 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.044122 30865 raft_consensus.cc:1261] T e109ef8f5d764df7969642dc6de361a2 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.044158 354 raft_consensus.cc:1261] T e109ef8f5d764df7969642dc6de361a2 P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.044485 2151 consensus_queue.cc:1035] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.044631 940 consensus_queue.cc:1035] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.044764 2149 consensus_queue.cc:1035] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.046506 2151 raft_consensus.cc:2932] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.046895 30865 raft_consensus.cc:2932] T e109ef8f5d764df7969642dc6de361a2 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.047865 354 raft_consensus.cc:2932] T e109ef8f5d764df7969642dc6de361a2 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.048291 31134 raft_consensus.cc:2932] T e109ef8f5d764df7969642dc6de361a2 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.049441 30707 catalog_manager.cc:5484] T e109ef8f5d764df7969642dc6de361a2 P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.054515 31001 consensus_queue.cc:237] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } }
I20240502 14:00:38.055684 354 raft_consensus.cc:1261] T e109ef8f5d764df7969642dc6de361a2 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.055897 30865 raft_consensus.cc:1261] T e109ef8f5d764df7969642dc6de361a2 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.056258 2149 consensus_queue.cc:1035] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.056700 2149 consensus_queue.cc:1035] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.057722 1538 raft_consensus.cc:2932] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.058094 30865 raft_consensus.cc:2932] T e109ef8f5d764df7969642dc6de361a2 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.058938 30700 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet e109ef8f5d764df7969642dc6de361a2 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:38.059662 354 raft_consensus.cc:2932] T e109ef8f5d764df7969642dc6de361a2 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.059774 30709 catalog_manager.cc:5484] T e109ef8f5d764df7969642dc6de361a2 P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:38.061964 31115 tablet_service.cc:1514] Processing DeleteTablet for tablet e109ef8f5d764df7969642dc6de361a2 with delete_type TABLET_DATA_TOMBSTONED (TS 8c61042437444f5f92c76109dfb229fc not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:42546
I20240502 14:00:38.062383 2159 tablet_replica.cc:330] T e109ef8f5d764df7969642dc6de361a2 P 8c61042437444f5f92c76109dfb229fc: stopping tablet replica
I20240502 14:00:38.062635 2159 raft_consensus.cc:2229] T e109ef8f5d764df7969642dc6de361a2 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:38.062839 2159 raft_consensus.cc:2258] T e109ef8f5d764df7969642dc6de361a2 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:38.063843 2159 ts_tablet_manager.cc:1875] T e109ef8f5d764df7969642dc6de361a2 P 8c61042437444f5f92c76109dfb229fc: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:38.065577 2159 ts_tablet_manager.cc:1888] T e109ef8f5d764df7969642dc6de361a2 P 8c61042437444f5f92c76109dfb229fc: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:38.065662 2159 log.cc:1196] T e109ef8f5d764df7969642dc6de361a2 P 8c61042437444f5f92c76109dfb229fc: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/wals/e109ef8f5d764df7969642dc6de361a2
I20240502 14:00:38.066157 30701 catalog_manager.cc:4830] TS 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193): tablet e109ef8f5d764df7969642dc6de361a2 (table rebalance_test_table_5 [id=b000c5e9f8984019bef07f52b5bff49d]) successfully deleted
I20240502 14:00:38.069306 2149 raft_consensus.cc:1050] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929: attempting to promote NON_VOTER 3daf9e0837c74a919c89e686597ff56b to VOTER
I20240502 14:00:38.069682 2149 consensus_queue.cc:237] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:38.070827 31134 raft_consensus.cc:1261] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.070828 31265 raft_consensus.cc:1261] T 8ec88a191b8d4188985b5e3c91a2a031 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.070887 488 raft_consensus.cc:1261] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.071246 2151 consensus_queue.cc:1035] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.071404 940 consensus_queue.cc:1035] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.071539 1538 consensus_queue.cc:1035] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.073844 1538 raft_consensus.cc:2932] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.074234 31134 raft_consensus.cc:2932] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.074501 488 raft_consensus.cc:2932] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.076197 31265 raft_consensus.cc:2932] T 8ec88a191b8d4188985b5e3c91a2a031 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.076530 30710 catalog_manager.cc:5484] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:38.079123 31001 consensus_queue.cc:237] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:38.080127 31134 raft_consensus.cc:1261] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.080529 488 raft_consensus.cc:1261] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.080513 2151 consensus_queue.cc:1035] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.080938 2151 consensus_queue.cc:1035] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.081935 2149 raft_consensus.cc:2932] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.082679 488 raft_consensus.cc:2932] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.083043 30700 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 8ec88a191b8d4188985b5e3c91a2a031 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:38.083333 31134 raft_consensus.cc:2932] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.083801 30707 catalog_manager.cc:5484] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:38.086036 31246 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ec88a191b8d4188985b5e3c91a2a031 with delete_type TABLET_DATA_TOMBSTONED (TS 14753c93125941acaad7283edc669052 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:53826
I20240502 14:00:38.086524 2164 tablet_replica.cc:330] T 8ec88a191b8d4188985b5e3c91a2a031 P 14753c93125941acaad7283edc669052: stopping tablet replica
I20240502 14:00:38.086759 2164 raft_consensus.cc:2229] T 8ec88a191b8d4188985b5e3c91a2a031 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:38.086992 2164 raft_consensus.cc:2258] T 8ec88a191b8d4188985b5e3c91a2a031 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:38.087913 2164 ts_tablet_manager.cc:1875] T 8ec88a191b8d4188985b5e3c91a2a031 P 14753c93125941acaad7283edc669052: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:38.089672 2164 ts_tablet_manager.cc:1888] T 8ec88a191b8d4188985b5e3c91a2a031 P 14753c93125941acaad7283edc669052: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:38.089740 2164 log.cc:1196] T 8ec88a191b8d4188985b5e3c91a2a031 P 14753c93125941acaad7283edc669052: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/8ec88a191b8d4188985b5e3c91a2a031
I20240502 14:00:38.090171 30701 catalog_manager.cc:4830] TS 14753c93125941acaad7283edc669052 (127.24.66.68:37029): tablet 8ec88a191b8d4188985b5e3c91a2a031 (table rebalance_test_table_1 [id=a4f4cd48a9854811903166c61a300083]) successfully deleted
I20240502 14:00:38.107041 620 raft_consensus.cc:1203] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:38.146055 2176 raft_consensus.cc:1050] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f: attempting to promote NON_VOTER 2bb442cde32f426bb14791852dd81802 to VOTER
I20240502 14:00:38.146730 2176 consensus_queue.cc:237] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } }
I20240502 14:00:38.148103 31134 raft_consensus.cc:1261] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.148406 31001 raft_consensus.cc:1261] T 8da28cc4aef043baaa2282f84afd4abf P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.148777 620 raft_consensus.cc:1261] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.149178 2183 consensus_queue.cc:1035] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.149353 2175 consensus_queue.cc:1035] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.149538 926 consensus_queue.cc:1035] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.152907 2175 raft_consensus.cc:2932] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.153424 31134 raft_consensus.cc:2932] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.154686 31002 raft_consensus.cc:2932] T 8da28cc4aef043baaa2282f84afd4abf P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.155071 620 raft_consensus.cc:2932] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.155882 30709 catalog_manager.cc:5484] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:38.159162 30865 consensus_queue.cc:237] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } }
I20240502 14:00:38.160663 31134 raft_consensus.cc:1261] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.161020 926 consensus_queue.cc:1035] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.161068 620 raft_consensus.cc:1261] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.161549 926 consensus_queue.cc:1035] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.163013 2183 raft_consensus.cc:2932] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.163478 31134 raft_consensus.cc:2932] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.164422 30698 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 8da28cc4aef043baaa2282f84afd4abf with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:38.164789 620 raft_consensus.cc:2932] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.165278 30715 catalog_manager.cc:5484] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index 5 to 6, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:38.167616 30982 tablet_service.cc:1514] Processing DeleteTablet for tablet 8da28cc4aef043baaa2282f84afd4abf with delete_type TABLET_DATA_TOMBSTONED (TS 1fab10dcfb8542fca2b965fed6c67929 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:58604
I20240502 14:00:38.168089 2197 tablet_replica.cc:330] T 8da28cc4aef043baaa2282f84afd4abf P 1fab10dcfb8542fca2b965fed6c67929: stopping tablet replica
I20240502 14:00:38.168444 2197 raft_consensus.cc:2229] T 8da28cc4aef043baaa2282f84afd4abf P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:38.168676 2197 raft_consensus.cc:2258] T 8da28cc4aef043baaa2282f84afd4abf P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:38.169997 2197 ts_tablet_manager.cc:1875] T 8da28cc4aef043baaa2282f84afd4abf P 1fab10dcfb8542fca2b965fed6c67929: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:38.172652 2197 ts_tablet_manager.cc:1888] T 8da28cc4aef043baaa2282f84afd4abf P 1fab10dcfb8542fca2b965fed6c67929: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:38.172734 2197 log.cc:1196] T 8da28cc4aef043baaa2282f84afd4abf P 1fab10dcfb8542fca2b965fed6c67929: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/wals/8da28cc4aef043baaa2282f84afd4abf
I20240502 14:00:38.173223 30700 catalog_manager.cc:4830] TS 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395): tablet 8da28cc4aef043baaa2282f84afd4abf (table rebalance_test_table_7 [id=2c4adac4a0a74af9bacbd4987a8a8d39]) successfully deleted
I20240502 14:00:38.185312 32691 raft_consensus.cc:1203] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:38.246124 2151 raft_consensus.cc:1050] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929: attempting to promote NON_VOTER f769c13ece0b40179ba120dbebbaba98 to VOTER
I20240502 14:00:38.246531 2151 consensus_queue.cc:237] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } }
I20240502 14:00:38.247764 31265 raft_consensus.cc:1261] T 1f3459ebf25f45dd810046420f5dcd81 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.247821 31134 raft_consensus.cc:1261] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.247931 354 raft_consensus.cc:1261] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.248232 940 consensus_queue.cc:1035] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.248597 940 consensus_queue.cc:1035] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.248896 940 consensus_queue.cc:1035] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.249670 30865 consensus_queue.cc:237] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:38.251371 31266 raft_consensus.cc:1261] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:38.251657 31134 raft_consensus.cc:1261] T 5a3c9c93437c43b1a9c0326f2880a04e P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:38.251749 1538 raft_consensus.cc:2932] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.252172 926 consensus_queue.cc:1035] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:38.252121 354 raft_consensus.cc:2932] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.252512 926 consensus_queue.cc:1035] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:38.254622 30707 catalog_manager.cc:5484] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:38.254717 31266 raft_consensus.cc:2932] T 1f3459ebf25f45dd810046420f5dcd81 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.256570 926 raft_consensus.cc:2932] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:38.256973 32250 raft_consensus.cc:1050] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc: attempting to promote NON_VOTER 2bb442cde32f426bb14791852dd81802 to VOTER
I20240502 14:00:38.257500 32250 consensus_queue.cc:237] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } }
I20240502 14:00:38.258178 31002 consensus_queue.cc:237] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } }
I20240502 14:00:38.258988 31266 raft_consensus.cc:1261] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.258970 31134 raft_consensus.cc:2932] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.259336 2289 consensus_queue.cc:1035] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.260143 354 raft_consensus.cc:1261] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.260155 620 raft_consensus.cc:1261] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.260589 2149 consensus_queue.cc:1035] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.260936 31002 raft_consensus.cc:1261] T cc64ea7db2064251bed364b800b36b87 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.261118 32250 consensus_queue.cc:1035] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.261456 31134 raft_consensus.cc:1261] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.262077 2151 consensus_queue.cc:1035] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.262512 620 raft_consensus.cc:1203] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:38.263206 31266 raft_consensus.cc:2932] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:38.263974 2292 consensus_queue.cc:1035] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.264014 31134 raft_consensus.cc:2932] T 5a3c9c93437c43b1a9c0326f2880a04e P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:38.265209 940 raft_consensus.cc:2932] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.265228 30709 catalog_manager.cc:5484] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:38.266559 2289 raft_consensus.cc:2932] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.267247 354 raft_consensus.cc:2932] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.268153 30700 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 1f3459ebf25f45dd810046420f5dcd81 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:38.268079 31266 raft_consensus.cc:2932] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.268950 2149 raft_consensus.cc:1050] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929: attempting to promote NON_VOTER f769c13ece0b40179ba120dbebbaba98 to VOTER
I20240502 14:00:38.269299 2149 consensus_queue.cc:237] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } }
I20240502 14:00:38.269364 30707 catalog_manager.cc:5484] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 reported cstate change: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.270108 619 raft_consensus.cc:2932] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.270884 354 raft_consensus.cc:1261] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.270947 31134 raft_consensus.cc:2932] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.272128 31002 raft_consensus.cc:2932] T cc64ea7db2064251bed364b800b36b87 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.272455 31246 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f3459ebf25f45dd810046420f5dcd81 with delete_type TABLET_DATA_TOMBSTONED (TS 14753c93125941acaad7283edc669052 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:53826
I20240502 14:00:38.272663 2164 tablet_replica.cc:330] T 1f3459ebf25f45dd810046420f5dcd81 P 14753c93125941acaad7283edc669052: stopping tablet replica
I20240502 14:00:38.272794 2164 raft_consensus.cc:2229] T 1f3459ebf25f45dd810046420f5dcd81 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:38.272912 2164 raft_consensus.cc:2258] T 1f3459ebf25f45dd810046420f5dcd81 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:38.273036 2151 consensus_queue.cc:1035] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.273061 30707 catalog_manager.cc:5484] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802 reported cstate change: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.273991 30865 raft_consensus.cc:1261] T be25ac1c6e7242519ff636242718cb79 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.274705 2164 ts_tablet_manager.cc:1875] T 1f3459ebf25f45dd810046420f5dcd81 P 14753c93125941acaad7283edc669052: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:38.274855 31266 raft_consensus.cc:1261] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.276499 2164 ts_tablet_manager.cc:1888] T 1f3459ebf25f45dd810046420f5dcd81 P 14753c93125941acaad7283edc669052: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:38.276564 2164 log.cc:1196] T 1f3459ebf25f45dd810046420f5dcd81 P 14753c93125941acaad7283edc669052: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/1f3459ebf25f45dd810046420f5dcd81
I20240502 14:00:38.277268 30701 catalog_manager.cc:4830] TS 14753c93125941acaad7283edc669052 (127.24.66.68:37029): tablet 1f3459ebf25f45dd810046420f5dcd81 (table rebalance_test_table_9 [id=5f28187b5b7a472f8429ed627f138f76]) successfully deleted
I20240502 14:00:38.278267 1538 consensus_queue.cc:1035] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.278577 1538 consensus_queue.cc:1035] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.279294 31134 consensus_queue.cc:237] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } }
W20240502 14:00:38.280553 30797 consensus_peers.cc:485] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f -> Peer f769c13ece0b40179ba120dbebbaba98 (127.24.66.70:38291): Couldn't send request to peer f769c13ece0b40179ba120dbebbaba98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5a3c9c93437c43b1a9c0326f2880a04e. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:38.281078 31265 raft_consensus.cc:1261] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.281087 619 raft_consensus.cc:1261] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.281545 32250 consensus_queue.cc:1035] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.282570 940 raft_consensus.cc:2932] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.282804 32250 consensus_queue.cc:1035] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.283128 30865 raft_consensus.cc:2932] T be25ac1c6e7242519ff636242718cb79 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.284767 354 raft_consensus.cc:2932] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.285691 619 raft_consensus.cc:2932] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.285889 30709 catalog_manager.cc:5484] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:38.286996 32250 raft_consensus.cc:2932] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.287654 30710 catalog_manager.cc:5484] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802 reported cstate change: config changed from index 5 to 6, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.288475 30701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet cc64ea7db2064251bed364b800b36b87 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:38.289134 488 raft_consensus.cc:1203] T 2051076a81274b688554c218c107ee1c P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:38.289242 619 raft_consensus.cc:1203] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:38.289925 31265 raft_consensus.cc:2932] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.292645 31266 raft_consensus.cc:2932] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.293967 31002 consensus_queue.cc:237] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } }
I20240502 14:00:38.295663 354 raft_consensus.cc:1261] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.296016 2149 consensus_queue.cc:1035] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.296097 30982 tablet_service.cc:1514] Processing DeleteTablet for tablet cc64ea7db2064251bed364b800b36b87 with delete_type TABLET_DATA_TOMBSTONED (TS 1fab10dcfb8542fca2b965fed6c67929 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:58604
I20240502 14:00:38.296290 2197 tablet_replica.cc:330] T cc64ea7db2064251bed364b800b36b87 P 1fab10dcfb8542fca2b965fed6c67929: stopping tablet replica
I20240502 14:00:38.296458 2197 raft_consensus.cc:2229] T cc64ea7db2064251bed364b800b36b87 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:38.296586 2197 raft_consensus.cc:2258] T cc64ea7db2064251bed364b800b36b87 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:38.297302 31266 raft_consensus.cc:1261] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.297416 2151 raft_consensus.cc:2932] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.297714 354 raft_consensus.cc:2932] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.297971 2197 ts_tablet_manager.cc:1875] T cc64ea7db2064251bed364b800b36b87 P 1fab10dcfb8542fca2b965fed6c67929: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:38.300010 2197 ts_tablet_manager.cc:1888] T cc64ea7db2064251bed364b800b36b87 P 1fab10dcfb8542fca2b965fed6c67929: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:38.300084 2197 log.cc:1196] T cc64ea7db2064251bed364b800b36b87 P 1fab10dcfb8542fca2b965fed6c67929: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/wals/cc64ea7db2064251bed364b800b36b87
I20240502 14:00:38.300258 30713 catalog_manager.cc:5484] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98 reported cstate change: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.300791 2149 consensus_queue.cc:1035] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.301582 30700 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet be25ac1c6e7242519ff636242718cb79 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:38.302240 31134 consensus_queue.cc:237] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:38.302352 30843 tablet_service.cc:1514] Processing DeleteTablet for tablet be25ac1c6e7242519ff636242718cb79 with delete_type TABLET_DATA_TOMBSTONED (TS ad5befacd63742a1b5b69fdacdb8cf8f not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45544
I20240502 14:00:38.302875 2304 tablet_replica.cc:330] T be25ac1c6e7242519ff636242718cb79 P ad5befacd63742a1b5b69fdacdb8cf8f: stopping tablet replica
I20240502 14:00:38.303221 2304 raft_consensus.cc:2229] T be25ac1c6e7242519ff636242718cb79 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:38.303454 2304 raft_consensus.cc:2258] T be25ac1c6e7242519ff636242718cb79 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Raft consensus is shut down!
W20240502 14:00:38.303988 31067 consensus_peers.cc:485] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc -> Peer 2bb442cde32f426bb14791852dd81802 (127.24.66.72:34475): Couldn't send request to peer 2bb442cde32f426bb14791852dd81802. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 66e2378d5c0f400299a170df8a939577. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:38.304566 31265 raft_consensus.cc:1261] T 66e2378d5c0f400299a170df8a939577 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:38.304711 2304 ts_tablet_manager.cc:1875] T be25ac1c6e7242519ff636242718cb79 P ad5befacd63742a1b5b69fdacdb8cf8f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:38.304924 2292 consensus_queue.cc:1035] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:38.305289 31266 raft_consensus.cc:2932] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.307503 2304 ts_tablet_manager.cc:1888] T be25ac1c6e7242519ff636242718cb79 P ad5befacd63742a1b5b69fdacdb8cf8f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:38.307538 30865 raft_consensus.cc:1261] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:38.307641 2304 log.cc:1196] T be25ac1c6e7242519ff636242718cb79 P ad5befacd63742a1b5b69fdacdb8cf8f: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/wals/be25ac1c6e7242519ff636242718cb79
I20240502 14:00:38.308015 2290 raft_consensus.cc:2932] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:38.308295 30698 catalog_manager.cc:4830] TS ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553): tablet be25ac1c6e7242519ff636242718cb79 (table rebalance_test_table_0 [id=a5535db3201345e78ac8467a4d987c08]) successfully deleted
I20240502 14:00:38.309039 2292 consensus_queue.cc:1035] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:38.309556 31265 raft_consensus.cc:2932] T 66e2378d5c0f400299a170df8a939577 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:38.310709 30700 catalog_manager.cc:4830] TS 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395): tablet cc64ea7db2064251bed364b800b36b87 (table rebalance_test_table_5 [id=b000c5e9f8984019bef07f52b5bff49d]) successfully deleted
I20240502 14:00:38.311429 30865 raft_consensus.cc:2932] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:38.311578 30713 catalog_manager.cc:5484] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:38.321370 31265 consensus_queue.cc:237] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:38.322631 31134 raft_consensus.cc:1261] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:38.322769 31002 raft_consensus.cc:1261] T 39239edeafb84b5caaa5dceea06fd4b0 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:00:38.322919 31201 consensus_peers.cc:485] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 -> Peer 3daf9e0837c74a919c89e686597ff56b (127.24.66.71:35161): Couldn't send request to peer 3daf9e0837c74a919c89e686597ff56b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39239edeafb84b5caaa5dceea06fd4b0. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:38.322968 2314 consensus_queue.cc:1035] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:38.323222 1620 consensus_queue.cc:1035] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:38.324663 2314 raft_consensus.cc:2932] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:38.325520 31002 raft_consensus.cc:2932] T 39239edeafb84b5caaa5dceea06fd4b0 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:38.325825 31134 raft_consensus.cc:2932] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:38.327837 30707 catalog_manager.cc:5484] T 39239edeafb84b5caaa5dceea06fd4b0 P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:38.328092 2317 raft_consensus.cc:1050] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052: attempting to promote NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 to VOTER
I20240502 14:00:38.328648 2317 consensus_queue.cc:237] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } }
I20240502 14:00:38.330315 32691 raft_consensus.cc:1261] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.330575 31002 raft_consensus.cc:1261] T 08588e44069d4044b2d09eb02d252f82 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.331017 2319 consensus_queue.cc:1035] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.331168 2148 consensus_queue.cc:1035] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.331800 31134 raft_consensus.cc:1261] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.332163 2148 consensus_queue.cc:1035] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.334537 2148 raft_consensus.cc:2932] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.335054 31002 raft_consensus.cc:2932] T 08588e44069d4044b2d09eb02d252f82 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.335836 2319 raft_consensus.cc:1050] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052: attempting to promote NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 to VOTER
I20240502 14:00:38.336222 2319 consensus_queue.cc:237] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } }
I20240502 14:00:38.336766 31134 raft_consensus.cc:2932] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.337584 32691 raft_consensus.cc:2932] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.338902 30707 catalog_manager.cc:5484] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.340373 31002 raft_consensus.cc:1261] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.341238 30865 raft_consensus.cc:1261] T 954fa01ea4d04842b0218b33de347aa1 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.341878 2317 consensus_queue.cc:1035] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.341922 32691 raft_consensus.cc:1261] T 954fa01ea4d04842b0218b33de347aa1 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.342151 31265 consensus_queue.cc:237] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } }
I20240502 14:00:38.342232 2317 consensus_queue.cc:1035] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.342813 2318 consensus_queue.cc:1035] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.343312 31134 raft_consensus.cc:1261] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.343430 32690 raft_consensus.cc:1261] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.343817 2317 consensus_queue.cc:1035] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.344130 2317 consensus_queue.cc:1035] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.345705 2317 raft_consensus.cc:2932] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.345983 32690 raft_consensus.cc:2932] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.347244 30701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 08588e44069d4044b2d09eb02d252f82 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:38.347471 31134 raft_consensus.cc:2932] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.348012 1620 raft_consensus.cc:2932] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.348510 32691 raft_consensus.cc:2932] T 954fa01ea4d04842b0218b33de347aa1 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.348733 30710 catalog_manager.cc:5484] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5 reported cstate change: config changed from index 5 to 6, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.349834 31002 raft_consensus.cc:2932] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.351572 30982 tablet_service.cc:1514] Processing DeleteTablet for tablet 08588e44069d4044b2d09eb02d252f82 with delete_type TABLET_DATA_TOMBSTONED (TS 1fab10dcfb8542fca2b965fed6c67929 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:58604
I20240502 14:00:38.351768 2197 tablet_replica.cc:330] T 08588e44069d4044b2d09eb02d252f82 P 1fab10dcfb8542fca2b965fed6c67929: stopping tablet replica
I20240502 14:00:38.351887 2197 raft_consensus.cc:2229] T 08588e44069d4044b2d09eb02d252f82 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:38.352011 2197 raft_consensus.cc:2258] T 08588e44069d4044b2d09eb02d252f82 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:38.352191 30865 raft_consensus.cc:2932] T 954fa01ea4d04842b0218b33de347aa1 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.352425 30713 catalog_manager.cc:5484] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.354521 2197 ts_tablet_manager.cc:1875] T 08588e44069d4044b2d09eb02d252f82 P 1fab10dcfb8542fca2b965fed6c67929: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:38.356783 2197 ts_tablet_manager.cc:1888] T 08588e44069d4044b2d09eb02d252f82 P 1fab10dcfb8542fca2b965fed6c67929: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:38.356848 2197 log.cc:1196] T 08588e44069d4044b2d09eb02d252f82 P 1fab10dcfb8542fca2b965fed6c67929: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/wals/08588e44069d4044b2d09eb02d252f82
I20240502 14:00:38.357318 30700 catalog_manager.cc:4830] TS 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395): tablet 08588e44069d4044b2d09eb02d252f82 (table rebalance_test_table_1 [id=a4f4cd48a9854811903166c61a300083]) successfully deleted
I20240502 14:00:38.359280 31265 consensus_queue.cc:237] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } }
I20240502 14:00:38.359459 354 raft_consensus.cc:1203] T 5702766b82f24e96a4cb5da5f051fc3f P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:38.360394 31002 raft_consensus.cc:1261] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.360821 2319 consensus_queue.cc:1035] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.361178 32691 raft_consensus.cc:1261] T 954fa01ea4d04842b0218b33de347aa1 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.361539 2319 consensus_queue.cc:1035] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.362906 2314 raft_consensus.cc:2932] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.363312 31002 raft_consensus.cc:2932] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.363447 32691 raft_consensus.cc:2932] T 954fa01ea4d04842b0218b33de347aa1 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.364084 30701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 954fa01ea4d04842b0218b33de347aa1 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:38.365446 30707 catalog_manager.cc:5484] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:38.367846 30843 tablet_service.cc:1514] Processing DeleteTablet for tablet 954fa01ea4d04842b0218b33de347aa1 with delete_type TABLET_DATA_TOMBSTONED (TS ad5befacd63742a1b5b69fdacdb8cf8f not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45544
I20240502 14:00:38.368116 2304 tablet_replica.cc:330] T 954fa01ea4d04842b0218b33de347aa1 P ad5befacd63742a1b5b69fdacdb8cf8f: stopping tablet replica
I20240502 14:00:38.368243 2304 raft_consensus.cc:2229] T 954fa01ea4d04842b0218b33de347aa1 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:38.368422 2304 raft_consensus.cc:2258] T 954fa01ea4d04842b0218b33de347aa1 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:38.369226 2304 ts_tablet_manager.cc:1875] T 954fa01ea4d04842b0218b33de347aa1 P ad5befacd63742a1b5b69fdacdb8cf8f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:38.371344 2304 ts_tablet_manager.cc:1888] T 954fa01ea4d04842b0218b33de347aa1 P ad5befacd63742a1b5b69fdacdb8cf8f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:38.371402 2304 log.cc:1196] T 954fa01ea4d04842b0218b33de347aa1 P ad5befacd63742a1b5b69fdacdb8cf8f: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/wals/954fa01ea4d04842b0218b33de347aa1
I20240502 14:00:38.372067 30698 catalog_manager.cc:4830] TS ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553): tablet 954fa01ea4d04842b0218b33de347aa1 (table rebalance_test_table_6 [id=99c461286f6a4e4f81cc54ae6fc11b21]) successfully deleted
I20240502 14:00:38.412482 2149 raft_consensus.cc:1050] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929: attempting to promote NON_VOTER f769c13ece0b40179ba120dbebbaba98 to VOTER
I20240502 14:00:38.412837 2149 consensus_queue.cc:237] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } }
I20240502 14:00:38.413990 30865 raft_consensus.cc:1261] T 42ecb8fd71404d7b9eb25af03d2200af P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.414098 354 raft_consensus.cc:1261] T 42ecb8fd71404d7b9eb25af03d2200af P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.414152 31134 raft_consensus.cc:1261] T 42ecb8fd71404d7b9eb25af03d2200af P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.414458 2149 consensus_queue.cc:1035] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.414616 940 consensus_queue.cc:1035] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.414742 2151 consensus_queue.cc:1035] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.417030 2149 raft_consensus.cc:2932] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.417271 354 raft_consensus.cc:2932] T 42ecb8fd71404d7b9eb25af03d2200af P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.418136 31134 raft_consensus.cc:2932] T 42ecb8fd71404d7b9eb25af03d2200af P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.419145 30865 raft_consensus.cc:2932] T 42ecb8fd71404d7b9eb25af03d2200af P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.419555 30715 catalog_manager.cc:5484] T 42ecb8fd71404d7b9eb25af03d2200af P f769c13ece0b40179ba120dbebbaba98 reported cstate change: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.422719 31002 consensus_queue.cc:237] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } }
I20240502 14:00:38.423758 30865 raft_consensus.cc:1261] T 42ecb8fd71404d7b9eb25af03d2200af P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.424167 2149 consensus_queue.cc:1035] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.424206 354 raft_consensus.cc:1261] T 42ecb8fd71404d7b9eb25af03d2200af P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.424865 1538 consensus_queue.cc:1035] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.425621 1538 raft_consensus.cc:2932] T 42ecb8fd71404d7b9eb25af03d2200af P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.426064 30865 raft_consensus.cc:2932] T 42ecb8fd71404d7b9eb25af03d2200af P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.426779 354 raft_consensus.cc:2932] T 42ecb8fd71404d7b9eb25af03d2200af P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.426901 30700 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 42ecb8fd71404d7b9eb25af03d2200af with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:38.428068 30708 catalog_manager.cc:5484] T 42ecb8fd71404d7b9eb25af03d2200af P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.430469 31115 tablet_service.cc:1514] Processing DeleteTablet for tablet 42ecb8fd71404d7b9eb25af03d2200af with delete_type TABLET_DATA_TOMBSTONED (TS 8c61042437444f5f92c76109dfb229fc not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:42546
I20240502 14:00:38.430644 2159 tablet_replica.cc:330] T 42ecb8fd71404d7b9eb25af03d2200af P 8c61042437444f5f92c76109dfb229fc: stopping tablet replica
I20240502 14:00:38.430765 2159 raft_consensus.cc:2229] T 42ecb8fd71404d7b9eb25af03d2200af P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:38.430905 2159 raft_consensus.cc:2258] T 42ecb8fd71404d7b9eb25af03d2200af P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:38.431468 2159 ts_tablet_manager.cc:1875] T 42ecb8fd71404d7b9eb25af03d2200af P 8c61042437444f5f92c76109dfb229fc: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:38.433265 2159 ts_tablet_manager.cc:1888] T 42ecb8fd71404d7b9eb25af03d2200af P 8c61042437444f5f92c76109dfb229fc: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:38.433326 2159 log.cc:1196] T 42ecb8fd71404d7b9eb25af03d2200af P 8c61042437444f5f92c76109dfb229fc: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/wals/42ecb8fd71404d7b9eb25af03d2200af
I20240502 14:00:38.433754 2175 raft_consensus.cc:1050] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f: attempting to promote NON_VOTER 3daf9e0837c74a919c89e686597ff56b to VOTER
I20240502 14:00:38.433804 30701 catalog_manager.cc:4830] TS 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193): tablet 42ecb8fd71404d7b9eb25af03d2200af (table rebalance_test_table_1 [id=a4f4cd48a9854811903166c61a300083]) successfully deleted
I20240502 14:00:38.434149 2175 consensus_queue.cc:237] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:38.435343 31001 raft_consensus.cc:1261] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.435711 31265 raft_consensus.cc:1261] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.435794 2183 consensus_queue.cc:1035] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.436630 926 consensus_queue.cc:1035] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.436758 2183 raft_consensus.cc:1050] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f: attempting to promote NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 to VOTER
I20240502 14:00:38.436909 488 raft_consensus.cc:1261] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.437057 2183 consensus_queue.cc:237] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } }
I20240502 14:00:38.437304 2176 consensus_queue.cc:1035] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.438509 32691 raft_consensus.cc:1261] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.439019 31002 raft_consensus.cc:1261] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.439352 2175 consensus_queue.cc:1035] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.439533 31266 raft_consensus.cc:1261] T aaca0594b68146cdbd18946f123693be P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.439813 2176 raft_consensus.cc:2932] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.440630 31002 raft_consensus.cc:2932] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.441553 2175 consensus_queue.cc:1035] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.442159 31266 raft_consensus.cc:2932] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.443234 488 raft_consensus.cc:2932] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.444830 30707 catalog_manager.cc:5484] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.445952 2183 consensus_queue.cc:1035] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.446311 926 raft_consensus.cc:2932] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.447031 32691 raft_consensus.cc:2932] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.447705 31265 raft_consensus.cc:2932] T aaca0594b68146cdbd18946f123693be P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.447922 31002 raft_consensus.cc:2932] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.453145 30713 catalog_manager.cc:5484] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5 reported cstate change: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.457994 30865 consensus_queue.cc:237] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:38.459000 488 raft_consensus.cc:1261] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.459127 31002 raft_consensus.cc:1261] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.459327 2183 consensus_queue.cc:1035] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.460134 2183 consensus_queue.cc:1035] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.461072 2183 raft_consensus.cc:2932] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.462157 30698 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet b5b3cf5fbe3d4d999cbf910c8e0ca62e with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:38.462800 488 raft_consensus.cc:2932] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.463162 30715 catalog_manager.cc:5484] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:38.465646 31002 raft_consensus.cc:2932] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.468504 31246 tablet_service.cc:1514] Processing DeleteTablet for tablet b5b3cf5fbe3d4d999cbf910c8e0ca62e with delete_type TABLET_DATA_TOMBSTONED (TS 14753c93125941acaad7283edc669052 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:53826
I20240502 14:00:38.468986 2164 tablet_replica.cc:330] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 14753c93125941acaad7283edc669052: stopping tablet replica
I20240502 14:00:38.469094 2164 raft_consensus.cc:2229] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:38.469192 2164 raft_consensus.cc:2258] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:38.469833 2164 ts_tablet_manager.cc:1875] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 14753c93125941acaad7283edc669052: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:38.471633 2164 ts_tablet_manager.cc:1888] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 14753c93125941acaad7283edc669052: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:38.471716 2164 log.cc:1196] T b5b3cf5fbe3d4d999cbf910c8e0ca62e P 14753c93125941acaad7283edc669052: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/b5b3cf5fbe3d4d999cbf910c8e0ca62e
I20240502 14:00:38.472183 30701 catalog_manager.cc:4830] TS 14753c93125941acaad7283edc669052 (127.24.66.68:37029): tablet b5b3cf5fbe3d4d999cbf910c8e0ca62e (table rebalance_test_table_6 [id=99c461286f6a4e4f81cc54ae6fc11b21]) successfully deleted
I20240502 14:00:38.491766 30865 consensus_queue.cc:237] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:38.492883 31002 raft_consensus.cc:1261] T efd8e3b804cf41bebf5e629b65571979 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:00:38.493006 30794 consensus_peers.cc:485] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f -> Peer 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69:35027): Couldn't send request to peer 2716cee98b1b4c4591a40e1f466520e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: efd8e3b804cf41bebf5e629b65571979. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:38.493041 31134 raft_consensus.cc:1261] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:38.493232 2183 consensus_queue.cc:1035] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:38.493407 2176 consensus_queue.cc:1035] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:38.494962 2175 raft_consensus.cc:2932] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:38.495514 31134 raft_consensus.cc:2932] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:38.496752 31002 raft_consensus.cc:2932] T efd8e3b804cf41bebf5e629b65571979 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:38.498723 30707 catalog_manager.cc:5484] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index -1 to 4, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:38.501269 940 raft_consensus.cc:1050] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929: attempting to promote NON_VOTER 3daf9e0837c74a919c89e686597ff56b to VOTER
I20240502 14:00:38.501619 940 consensus_queue.cc:237] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:38.502964 31134 raft_consensus.cc:1261] T c43ff8fc0b4f4f799a6f1bbd677166da P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.503032 488 raft_consensus.cc:1261] T c43ff8fc0b4f4f799a6f1bbd677166da P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.503062 31266 raft_consensus.cc:1261] T c43ff8fc0b4f4f799a6f1bbd677166da P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.503371 2149 consensus_queue.cc:1035] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.503593 2151 consensus_queue.cc:1035] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.503924 1538 consensus_queue.cc:1035] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.505735 2149 raft_consensus.cc:2932] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.505944 31266 consensus_queue.cc:237] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:38.506140 31134 raft_consensus.cc:2932] T c43ff8fc0b4f4f799a6f1bbd677166da P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.506199 31265 raft_consensus.cc:2932] T c43ff8fc0b4f4f799a6f1bbd677166da P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.507949 488 raft_consensus.cc:2932] T c43ff8fc0b4f4f799a6f1bbd677166da P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.508080 30865 raft_consensus.cc:1261] T da2f3b3481da476d98c97bf35f055c2e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:38.508319 30715 catalog_manager.cc:5484] T c43ff8fc0b4f4f799a6f1bbd677166da P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.508466 2314 consensus_queue.cc:1035] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:38.508800 31134 raft_consensus.cc:1261] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:38.509433 2314 consensus_queue.cc:1035] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:38.512033 2148 raft_consensus.cc:2932] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:38.512513 30865 raft_consensus.cc:2932] T da2f3b3481da476d98c97bf35f055c2e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:38.512528 31134 raft_consensus.cc:2932] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
W20240502 14:00:38.513437 31201 consensus_peers.cc:485] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 -> Peer f769c13ece0b40179ba120dbebbaba98 (127.24.66.70:38291): Couldn't send request to peer f769c13ece0b40179ba120dbebbaba98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: da2f3b3481da476d98c97bf35f055c2e. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:38.513939 31002 consensus_queue.cc:237] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:38.515129 488 raft_consensus.cc:1261] T c43ff8fc0b4f4f799a6f1bbd677166da P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.515430 2151 consensus_queue.cc:1035] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.515827 31265 raft_consensus.cc:1261] T c43ff8fc0b4f4f799a6f1bbd677166da P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.516181 2151 consensus_queue.cc:1035] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.516392 30707 catalog_manager.cc:5484] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:38.516894 940 raft_consensus.cc:2932] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.517253 488 raft_consensus.cc:2932] T c43ff8fc0b4f4f799a6f1bbd677166da P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.518253 30700 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet c43ff8fc0b4f4f799a6f1bbd677166da with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:38.519219 30708 catalog_manager.cc:5484] T c43ff8fc0b4f4f799a6f1bbd677166da P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:38.520691 31265 raft_consensus.cc:2932] T c43ff8fc0b4f4f799a6f1bbd677166da P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.521639 31115 tablet_service.cc:1514] Processing DeleteTablet for tablet c43ff8fc0b4f4f799a6f1bbd677166da with delete_type TABLET_DATA_TOMBSTONED (TS 8c61042437444f5f92c76109dfb229fc not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:42546
I20240502 14:00:38.521842 2159 tablet_replica.cc:330] T c43ff8fc0b4f4f799a6f1bbd677166da P 8c61042437444f5f92c76109dfb229fc: stopping tablet replica
I20240502 14:00:38.521960 2159 raft_consensus.cc:2229] T c43ff8fc0b4f4f799a6f1bbd677166da P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:38.522075 2159 raft_consensus.cc:2258] T c43ff8fc0b4f4f799a6f1bbd677166da P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:38.522697 2159 ts_tablet_manager.cc:1875] T c43ff8fc0b4f4f799a6f1bbd677166da P 8c61042437444f5f92c76109dfb229fc: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:38.525076 2159 ts_tablet_manager.cc:1888] T c43ff8fc0b4f4f799a6f1bbd677166da P 8c61042437444f5f92c76109dfb229fc: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:38.525149 2159 log.cc:1196] T c43ff8fc0b4f4f799a6f1bbd677166da P 8c61042437444f5f92c76109dfb229fc: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/wals/c43ff8fc0b4f4f799a6f1bbd677166da
I20240502 14:00:38.525607 30701 catalog_manager.cc:4830] TS 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193): tablet c43ff8fc0b4f4f799a6f1bbd677166da (table rebalance_test_table_9 [id=5f28187b5b7a472f8429ed627f138f76]) successfully deleted
I20240502 14:00:38.526454 30865 consensus_queue.cc:237] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:38.527463 31265 raft_consensus.cc:1261] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:38.527539 31002 raft_consensus.cc:1261] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:00:38.527742 30794 consensus_peers.cc:485] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f -> Peer 2bb442cde32f426bb14791852dd81802 (127.24.66.72:34475): Couldn't send request to peer 2bb442cde32f426bb14791852dd81802. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4a12b8eaab7943c3bc9ee48c2a2307d0. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:38.527839 2183 consensus_queue.cc:1035] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:38.527976 926 consensus_queue.cc:1035] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:38.529574 2176 raft_consensus.cc:2932] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:38.530002 31002 raft_consensus.cc:2932] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:38.531023 31265 raft_consensus.cc:2932] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:38.531989 30708 catalog_manager.cc:5484] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:38.536057 2317 raft_consensus.cc:1050] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052: attempting to promote NON_VOTER 3daf9e0837c74a919c89e686597ff56b to VOTER
I20240502 14:00:38.536372 2317 consensus_queue.cc:237] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:38.537583 31002 raft_consensus.cc:1261] T 46b0a06884ad48da9b060d2810d5c782 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.537664 30865 raft_consensus.cc:1261] T 46b0a06884ad48da9b060d2810d5c782 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.537992 2314 consensus_queue.cc:1035] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.538065 488 raft_consensus.cc:1261] T 46b0a06884ad48da9b060d2810d5c782 P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.538123 1620 consensus_queue.cc:1035] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.538395 2319 consensus_queue.cc:1035] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.540439 2319 raft_consensus.cc:2932] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.540968 30865 raft_consensus.cc:2932] T 46b0a06884ad48da9b060d2810d5c782 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.541184 31002 raft_consensus.cc:2932] T 46b0a06884ad48da9b060d2810d5c782 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.542439 488 raft_consensus.cc:2932] T 46b0a06884ad48da9b060d2810d5c782 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.543447 30707 catalog_manager.cc:5484] T 46b0a06884ad48da9b060d2810d5c782 P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.546238 31265 consensus_queue.cc:237] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:38.547155 488 raft_consensus.cc:1261] T 46b0a06884ad48da9b060d2810d5c782 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.547210 31002 raft_consensus.cc:1261] T 46b0a06884ad48da9b060d2810d5c782 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.548180 2317 consensus_queue.cc:1035] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.548329 2318 consensus_queue.cc:1035] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.548781 2183 raft_consensus.cc:1050] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f: attempting to promote NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 to VOTER
I20240502 14:00:38.549110 2183 consensus_queue.cc:237] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } }
I20240502 14:00:38.549944 2318 raft_consensus.cc:2932] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.550298 31002 raft_consensus.cc:2932] T 46b0a06884ad48da9b060d2810d5c782 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.550531 32691 raft_consensus.cc:1261] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.550535 31134 raft_consensus.cc:1261] T d0409950632d4eeead8a826034b5a9e4 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.550912 2175 consensus_queue.cc:1035] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.551060 926 consensus_queue.cc:1035] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.551214 30701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 46b0a06884ad48da9b060d2810d5c782 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:38.552471 30713 catalog_manager.cc:5484] T 46b0a06884ad48da9b060d2810d5c782 P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:38.552731 31265 raft_consensus.cc:1261] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.553025 2175 consensus_queue.cc:1035] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.553803 488 raft_consensus.cc:2932] T 46b0a06884ad48da9b060d2810d5c782 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.555189 30843 tablet_service.cc:1514] Processing DeleteTablet for tablet 46b0a06884ad48da9b060d2810d5c782 with delete_type TABLET_DATA_TOMBSTONED (TS ad5befacd63742a1b5b69fdacdb8cf8f not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45544
I20240502 14:00:38.555502 2304 tablet_replica.cc:330] T 46b0a06884ad48da9b060d2810d5c782 P ad5befacd63742a1b5b69fdacdb8cf8f: stopping tablet replica
I20240502 14:00:38.555658 2304 raft_consensus.cc:2229] T 46b0a06884ad48da9b060d2810d5c782 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:38.555784 2304 raft_consensus.cc:2258] T 46b0a06884ad48da9b060d2810d5c782 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:38.557099 2304 ts_tablet_manager.cc:1875] T 46b0a06884ad48da9b060d2810d5c782 P ad5befacd63742a1b5b69fdacdb8cf8f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:38.559152 2304 ts_tablet_manager.cc:1888] T 46b0a06884ad48da9b060d2810d5c782 P ad5befacd63742a1b5b69fdacdb8cf8f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:38.559221 2304 log.cc:1196] T 46b0a06884ad48da9b060d2810d5c782 P ad5befacd63742a1b5b69fdacdb8cf8f: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/wals/46b0a06884ad48da9b060d2810d5c782
I20240502 14:00:38.559671 30698 catalog_manager.cc:4830] TS ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553): tablet 46b0a06884ad48da9b060d2810d5c782 (table rebalance_test_table_4 [id=4cd0d0b0efb44a2c966dd894b6e48601]) successfully deleted
I20240502 14:00:38.560663 926 raft_consensus.cc:2932] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.561053 32691 raft_consensus.cc:2932] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.561343 31134 raft_consensus.cc:2932] T d0409950632d4eeead8a826034b5a9e4 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.562863 31265 raft_consensus.cc:2932] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.563257 30708 catalog_manager.cc:5484] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:38.566891 30865 consensus_queue.cc:237] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } }
I20240502 14:00:38.568064 31265 raft_consensus.cc:1261] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.568145 32691 raft_consensus.cc:1261] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.568537 2175 consensus_queue.cc:1035] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.568725 2183 consensus_queue.cc:1035] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.570123 2183 raft_consensus.cc:2932] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.570466 32691 raft_consensus.cc:2932] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.571305 30698 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d0409950632d4eeead8a826034b5a9e4 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:38.571838 31265 raft_consensus.cc:2932] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.572898 30707 catalog_manager.cc:5484] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5 reported cstate change: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.575515 31115 tablet_service.cc:1514] Processing DeleteTablet for tablet d0409950632d4eeead8a826034b5a9e4 with delete_type TABLET_DATA_TOMBSTONED (TS 8c61042437444f5f92c76109dfb229fc not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:42546
I20240502 14:00:38.575731 2159 tablet_replica.cc:330] T d0409950632d4eeead8a826034b5a9e4 P 8c61042437444f5f92c76109dfb229fc: stopping tablet replica
I20240502 14:00:38.575877 2159 raft_consensus.cc:2229] T d0409950632d4eeead8a826034b5a9e4 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:38.576045 2159 raft_consensus.cc:2258] T d0409950632d4eeead8a826034b5a9e4 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:38.576761 2159 ts_tablet_manager.cc:1875] T d0409950632d4eeead8a826034b5a9e4 P 8c61042437444f5f92c76109dfb229fc: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:38.578632 2159 ts_tablet_manager.cc:1888] T d0409950632d4eeead8a826034b5a9e4 P 8c61042437444f5f92c76109dfb229fc: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:38.578711 2159 log.cc:1196] T d0409950632d4eeead8a826034b5a9e4 P 8c61042437444f5f92c76109dfb229fc: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/wals/d0409950632d4eeead8a826034b5a9e4
I20240502 14:00:38.579188 30701 catalog_manager.cc:4830] TS 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193): tablet d0409950632d4eeead8a826034b5a9e4 (table rebalance_test_table_7 [id=2c4adac4a0a74af9bacbd4987a8a8d39]) successfully deleted
I20240502 14:00:38.641647 32250 raft_consensus.cc:1050] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc: attempting to promote NON_VOTER 2bb442cde32f426bb14791852dd81802 to VOTER
I20240502 14:00:38.642022 32250 consensus_queue.cc:237] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } }
I20240502 14:00:38.643440 31265 raft_consensus.cc:1261] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.643523 30865 raft_consensus.cc:1261] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.644026 619 raft_consensus.cc:1261] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.644326 2289 consensus_queue.cc:1035] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.645421 2285 consensus_queue.cc:1035] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.645788 2285 consensus_queue.cc:1035] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.646085 926 raft_consensus.cc:1050] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f: attempting to promote NON_VOTER 2bb442cde32f426bb14791852dd81802 to VOTER
I20240502 14:00:38.646404 926 consensus_queue.cc:237] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } }
I20240502 14:00:38.647518 31265 raft_consensus.cc:1261] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.647629 31002 raft_consensus.cc:1261] T 342cec2187904922890444a7e5dd181b P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.647712 619 raft_consensus.cc:1261] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.647893 2175 consensus_queue.cc:1035] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.648061 2183 consensus_queue.cc:1035] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.648191 2176 consensus_queue.cc:1035] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.650149 2285 raft_consensus.cc:2932] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.650532 31265 raft_consensus.cc:2932] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.650972 2175 raft_consensus.cc:2932] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.652350 620 raft_consensus.cc:2932] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.653697 30713 catalog_manager.cc:5484] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.653982 30708 catalog_manager.cc:5484] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:38.654512 31002 raft_consensus.cc:2932] T 342cec2187904922890444a7e5dd181b P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.656672 619 raft_consensus.cc:2932] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.658135 31265 raft_consensus.cc:2932] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.658293 2289 raft_consensus.cc:1050] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc: attempting to promote NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 to VOTER
I20240502 14:00:38.658651 2289 consensus_queue.cc:237] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } }
I20240502 14:00:38.658950 30863 consensus_queue.cc:237] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } }
I20240502 14:00:38.660003 32691 raft_consensus.cc:1261] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.660555 30863 raft_consensus.cc:1261] T 2e07f0adca594453bad9433ed0017580 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.660935 31265 raft_consensus.cc:1261] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.661100 2292 consensus_queue.cc:1035] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.661373 2292 consensus_queue.cc:1035] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.661518 619 raft_consensus.cc:1261] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.661867 926 consensus_queue.cc:1035] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.662043 2176 consensus_queue.cc:1035] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.662526 31002 raft_consensus.cc:1261] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.662910 2292 consensus_queue.cc:1035] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.664031 926 raft_consensus.cc:2932] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.664404 31265 raft_consensus.cc:2932] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.665750 30698 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 342cec2187904922890444a7e5dd181b with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:38.666632 30708 catalog_manager.cc:5484] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index 5 to 6, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:38.666985 2292 raft_consensus.cc:2932] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.668572 619 raft_consensus.cc:2932] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.668871 31002 raft_consensus.cc:2932] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.669183 32691 raft_consensus.cc:2932] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.669546 30707 catalog_manager.cc:5484] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:38.672375 30982 tablet_service.cc:1514] Processing DeleteTablet for tablet 342cec2187904922890444a7e5dd181b with delete_type TABLET_DATA_TOMBSTONED (TS 1fab10dcfb8542fca2b965fed6c67929 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:58604
I20240502 14:00:38.672613 2197 tablet_replica.cc:330] T 342cec2187904922890444a7e5dd181b P 1fab10dcfb8542fca2b965fed6c67929: stopping tablet replica
I20240502 14:00:38.672756 2197 raft_consensus.cc:2229] T 342cec2187904922890444a7e5dd181b P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:38.672868 2197 raft_consensus.cc:2258] T 342cec2187904922890444a7e5dd181b P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:38.673060 30863 raft_consensus.cc:2932] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.674161 2197 ts_tablet_manager.cc:1875] T 342cec2187904922890444a7e5dd181b P 1fab10dcfb8542fca2b965fed6c67929: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:38.674206 30865 raft_consensus.cc:2932] T 2e07f0adca594453bad9433ed0017580 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.676373 2197 ts_tablet_manager.cc:1888] T 342cec2187904922890444a7e5dd181b P 1fab10dcfb8542fca2b965fed6c67929: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:38.676450 2197 log.cc:1196] T 342cec2187904922890444a7e5dd181b P 1fab10dcfb8542fca2b965fed6c67929: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/wals/342cec2187904922890444a7e5dd181b
I20240502 14:00:38.676995 30700 catalog_manager.cc:4830] TS 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395): tablet 342cec2187904922890444a7e5dd181b (table rebalance_test_table_9 [id=5f28187b5b7a472f8429ed627f138f76]) successfully deleted
I20240502 14:00:38.677961 31133 consensus_queue.cc:237] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } }
I20240502 14:00:38.678860 32691 raft_consensus.cc:1261] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.678860 31002 raft_consensus.cc:1261] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.679184 32250 consensus_queue.cc:1035] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.679343 2292 consensus_queue.cc:1035] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.679458 31133 consensus_queue.cc:237] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } }
I20240502 14:00:38.680503 619 raft_consensus.cc:1261] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.680517 30865 raft_consensus.cc:1261] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.680840 32250 consensus_queue.cc:1035] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.680770 2292 raft_consensus.cc:2932] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.681499 2285 consensus_queue.cc:1035] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.681603 31002 raft_consensus.cc:2932] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.682081 32691 raft_consensus.cc:2932] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.682881 32250 raft_consensus.cc:2932] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.683179 30701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2e07f0adca594453bad9433ed0017580 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:38.683586 30865 raft_consensus.cc:2932] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.684243 30710 catalog_manager.cc:5484] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5 reported cstate change: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.684531 30701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 03fb6c61ce4c4ccdafdc9ca7a40db4f5 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:38.686465 30713 catalog_manager.cc:5484] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.686236 619 raft_consensus.cc:2932] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.687633 30843 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e07f0adca594453bad9433ed0017580 with delete_type TABLET_DATA_TOMBSTONED (TS ad5befacd63742a1b5b69fdacdb8cf8f not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45544
I20240502 14:00:38.688234 2304 tablet_replica.cc:330] T 2e07f0adca594453bad9433ed0017580 P ad5befacd63742a1b5b69fdacdb8cf8f: stopping tablet replica
I20240502 14:00:38.688372 2304 raft_consensus.cc:2229] T 2e07f0adca594453bad9433ed0017580 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:38.688516 2304 raft_consensus.cc:2258] T 2e07f0adca594453bad9433ed0017580 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:38.689097 2304 ts_tablet_manager.cc:1875] T 2e07f0adca594453bad9433ed0017580 P ad5befacd63742a1b5b69fdacdb8cf8f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:38.689976 31246 tablet_service.cc:1514] Processing DeleteTablet for tablet 03fb6c61ce4c4ccdafdc9ca7a40db4f5 with delete_type TABLET_DATA_TOMBSTONED (TS 14753c93125941acaad7283edc669052 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:53826
I20240502 14:00:38.690140 2164 tablet_replica.cc:330] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 14753c93125941acaad7283edc669052: stopping tablet replica
I20240502 14:00:38.690250 2164 raft_consensus.cc:2229] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:38.690407 2164 raft_consensus.cc:2258] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:38.691001 2304 ts_tablet_manager.cc:1888] T 2e07f0adca594453bad9433ed0017580 P ad5befacd63742a1b5b69fdacdb8cf8f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:38.691030 2164 ts_tablet_manager.cc:1875] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 14753c93125941acaad7283edc669052: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:38.691066 2304 log.cc:1196] T 2e07f0adca594453bad9433ed0017580 P ad5befacd63742a1b5b69fdacdb8cf8f: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/wals/2e07f0adca594453bad9433ed0017580
I20240502 14:00:38.691557 30698 catalog_manager.cc:4830] TS ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553): tablet 2e07f0adca594453bad9433ed0017580 (table rebalance_test_table_8 [id=5cb834cd98904017a28fc7b595efc564]) successfully deleted
I20240502 14:00:38.692973 2164 ts_tablet_manager.cc:1888] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 14753c93125941acaad7283edc669052: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:38.693037 2164 log.cc:1196] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 14753c93125941acaad7283edc669052: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/03fb6c61ce4c4ccdafdc9ca7a40db4f5
I20240502 14:00:38.693535 30701 catalog_manager.cc:4830] TS 14753c93125941acaad7283edc669052 (127.24.66.68:37029): tablet 03fb6c61ce4c4ccdafdc9ca7a40db4f5 (table rebalance_test_table_2 [id=2109014949af4d09893b47c9b10ba7b5]) successfully deleted
I20240502 14:00:38.695251 31265 consensus_queue.cc:237] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:38.696301 31133 raft_consensus.cc:1261] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:00:38.696530 31201 consensus_peers.cc:485] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 -> Peer 3daf9e0837c74a919c89e686597ff56b (127.24.66.71:35161): Couldn't send request to peer 3daf9e0837c74a919c89e686597ff56b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f571119d6f624fe88bf1effeaf466085. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:38.696532 30865 raft_consensus.cc:1261] T f571119d6f624fe88bf1effeaf466085 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:38.696630 2317 consensus_queue.cc:1035] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:38.696866 2314 consensus_queue.cc:1035] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:38.698271 2589 ts_tablet_manager.cc:902] T 66e2378d5c0f400299a170df8a939577 P 2bb442cde32f426bb14791852dd81802: Initiating tablet copy from peer 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:38.698577 2148 raft_consensus.cc:2932] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:38.698952 2589 tablet_copy_client.cc:308] T 66e2378d5c0f400299a170df8a939577 P 2bb442cde32f426bb14791852dd81802: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.67:34193
I20240502 14:00:38.698909 31133 raft_consensus.cc:2932] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:38.699074 30865 raft_consensus.cc:2932] T f571119d6f624fe88bf1effeaf466085 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:38.699517 31155 tablet_copy_service.cc:140] P 8c61042437444f5f92c76109dfb229fc: Received BeginTabletCopySession request for tablet 66e2378d5c0f400299a170df8a939577 from peer 2bb442cde32f426bb14791852dd81802 ({username='slave'} at 127.24.66.72:50377)
I20240502 14:00:38.699594 31155 tablet_copy_service.cc:161] P 8c61042437444f5f92c76109dfb229fc: Beginning new tablet copy session on tablet 66e2378d5c0f400299a170df8a939577 from peer 2bb442cde32f426bb14791852dd81802 at {username='slave'} at 127.24.66.72:50377: session id = 2bb442cde32f426bb14791852dd81802-66e2378d5c0f400299a170df8a939577
I20240502 14:00:38.700553 31155 tablet_copy_source_session.cc:205] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:38.701548 2589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66e2378d5c0f400299a170df8a939577. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:38.701598 30707 catalog_manager.cc:5484] T f571119d6f624fe88bf1effeaf466085 P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:38.703953 2589 tablet_copy_client.cc:783] T 66e2378d5c0f400299a170df8a939577 P 2bb442cde32f426bb14791852dd81802: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:38.704099 2589 tablet_copy_client.cc:647] T 66e2378d5c0f400299a170df8a939577 P 2bb442cde32f426bb14791852dd81802: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:38.705070 2593 tablet_copy_client.cc:1002] T 66e2378d5c0f400299a170df8a939577 P 2bb442cde32f426bb14791852dd81802: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:38.709980 31133 consensus_queue.cc:237] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:38.711179 31265 raft_consensus.cc:1261] T c0b9178646764824b17cbd842fbb0ff2 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:38.711414 31002 raft_consensus.cc:1261] T c0b9178646764824b17cbd842fbb0ff2 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:38.711511 2290 consensus_queue.cc:1035] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:38.711755 2292 consensus_queue.cc:1035] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:38.713268 2290 raft_consensus.cc:2932] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:38.713666 31002 raft_consensus.cc:2932] T c0b9178646764824b17cbd842fbb0ff2 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:38.714115 31265 raft_consensus.cc:2932] T c0b9178646764824b17cbd842fbb0ff2 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:38.716149 30713 catalog_manager.cc:5484] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index -1 to 4, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20240502 14:00:38.719187 31069 consensus_peers.cc:485] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc -> Peer f769c13ece0b40179ba120dbebbaba98 (127.24.66.70:38291): Couldn't send request to peer f769c13ece0b40179ba120dbebbaba98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0b9178646764824b17cbd842fbb0ff2. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:38.722544 31002 consensus_queue.cc:237] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:38.723668 30865 raft_consensus.cc:1261] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:38.723750 31133 raft_consensus.cc:1261] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:00:38.723857 30935 consensus_peers.cc:485] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 -> Peer 3daf9e0837c74a919c89e686597ff56b (127.24.66.71:35161): Couldn't send request to peer 3daf9e0837c74a919c89e686597ff56b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2e02dc2a98d04bddafdd1b933d89b4dd. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:38.724051 1538 consensus_queue.cc:1035] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:38.724253 2151 consensus_queue.cc:1035] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:38.725904 940 raft_consensus.cc:2932] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:38.726290 31133 raft_consensus.cc:2932] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:38.726398 30865 raft_consensus.cc:2932] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:38.728628 30707 catalog_manager.cc:5484] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index -1 to 4, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:38.755613 1538 raft_consensus.cc:1050] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929: attempting to promote NON_VOTER 2bb442cde32f426bb14791852dd81802 to VOTER
I20240502 14:00:38.755995 1538 consensus_queue.cc:237] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } }
I20240502 14:00:38.757364 619 raft_consensus.cc:1261] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.757452 31133 raft_consensus.cc:1261] T 8c77a03ce6a142debba75e0d387aa8ec P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.757750 2151 consensus_queue.cc:1035] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.757879 31265 raft_consensus.cc:1261] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.758193 2149 consensus_queue.cc:1035] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.758337 2151 consensus_queue.cc:1035] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.760202 2149 raft_consensus.cc:2932] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.761276 31265 raft_consensus.cc:2932] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.762032 619 raft_consensus.cc:2932] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.763978 30710 catalog_manager.cc:5484] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802 reported cstate change: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.765004 31133 raft_consensus.cc:2932] T 8c77a03ce6a142debba75e0d387aa8ec P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.767621 31002 consensus_queue.cc:237] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } }
I20240502 14:00:38.768676 31266 raft_consensus.cc:1261] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.769253 2151 consensus_queue.cc:1035] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.769304 619 raft_consensus.cc:1261] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.769729 2151 consensus_queue.cc:1035] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.770752 1538 raft_consensus.cc:2932] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.771191 31266 raft_consensus.cc:2932] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.771492 619 raft_consensus.cc:2932] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:38.771977 30700 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 8c77a03ce6a142debba75e0d387aa8ec with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:38.773115 30715 catalog_manager.cc:5484] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:38.775029 31115 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c77a03ce6a142debba75e0d387aa8ec with delete_type TABLET_DATA_TOMBSTONED (TS 8c61042437444f5f92c76109dfb229fc not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:42546
I20240502 14:00:38.775192 2159 tablet_replica.cc:330] T 8c77a03ce6a142debba75e0d387aa8ec P 8c61042437444f5f92c76109dfb229fc: stopping tablet replica
I20240502 14:00:38.775312 2159 raft_consensus.cc:2229] T 8c77a03ce6a142debba75e0d387aa8ec P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:38.775442 2159 raft_consensus.cc:2258] T 8c77a03ce6a142debba75e0d387aa8ec P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:38.776048 2159 ts_tablet_manager.cc:1875] T 8c77a03ce6a142debba75e0d387aa8ec P 8c61042437444f5f92c76109dfb229fc: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:38.777907 2159 ts_tablet_manager.cc:1888] T 8c77a03ce6a142debba75e0d387aa8ec P 8c61042437444f5f92c76109dfb229fc: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:38.777971 2159 log.cc:1196] T 8c77a03ce6a142debba75e0d387aa8ec P 8c61042437444f5f92c76109dfb229fc: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/wals/8c77a03ce6a142debba75e0d387aa8ec
I20240502 14:00:38.778396 30701 catalog_manager.cc:4830] TS 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193): tablet 8c77a03ce6a142debba75e0d387aa8ec (table rebalance_test_table_3 [id=3fb9e83c8b7c4d90a375d14e792f650d]) successfully deleted
I20240502 14:00:38.817875 30865 consensus_queue.cc:237] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:38.819154 31266 raft_consensus.cc:1261] T 903f5d6c1faf4267b7f7d7dc32670a52 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:00:38.819320 30794 consensus_peers.cc:485] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f -> Peer 2bb442cde32f426bb14791852dd81802 (127.24.66.72:34475): Couldn't send request to peer 2bb442cde32f426bb14791852dd81802. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 903f5d6c1faf4267b7f7d7dc32670a52. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:38.819499 2183 consensus_queue.cc:1035] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:38.820540 31133 raft_consensus.cc:1261] T 903f5d6c1faf4267b7f7d7dc32670a52 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:00:38.820925 2183 consensus_queue.cc:1035] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:00:38.821286 2176 raft_consensus.cc:2932] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:38.821906 31133 raft_consensus.cc:2932] T 903f5d6c1faf4267b7f7d7dc32670a52 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:38.823999 30707 catalog_manager.cc:5484] T 903f5d6c1faf4267b7f7d7dc32670a52 P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:38.824246 31266 raft_consensus.cc:2932] T 903f5d6c1faf4267b7f7d7dc32670a52 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:38.837931 30865 consensus_queue.cc:237] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } }
I20240502 14:00:38.838888 2717 ts_tablet_manager.cc:902] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98: Initiating tablet copy from peer ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
I20240502 14:00:38.839018 31002 raft_consensus.cc:1261] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.839042 32691 raft_consensus.cc:1261] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.839365 2717 tablet_copy_client.cc:308] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.65:39553
I20240502 14:00:38.839434 926 consensus_queue.cc:1035] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.839735 926 consensus_queue.cc:1035] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.841220 926 raft_consensus.cc:2932] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.842080 32691 raft_consensus.cc:2932] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.842881 30698 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet aaca0594b68146cdbd18946f123693be with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:38.843704 30707 catalog_manager.cc:5484] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:38.845906 31002 raft_consensus.cc:2932] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:38.847019 30887 tablet_copy_service.cc:140] P ad5befacd63742a1b5b69fdacdb8cf8f: Received BeginTabletCopySession request for tablet 5a3c9c93437c43b1a9c0326f2880a04e from peer f769c13ece0b40179ba120dbebbaba98 ({username='slave'} at 127.24.66.70:52845)
I20240502 14:00:38.847092 30887 tablet_copy_service.cc:161] P ad5befacd63742a1b5b69fdacdb8cf8f: Beginning new tablet copy session on tablet 5a3c9c93437c43b1a9c0326f2880a04e from peer f769c13ece0b40179ba120dbebbaba98 at {username='slave'} at 127.24.66.70:52845: session id = f769c13ece0b40179ba120dbebbaba98-5a3c9c93437c43b1a9c0326f2880a04e
I20240502 14:00:38.847327 31246 tablet_service.cc:1514] Processing DeleteTablet for tablet aaca0594b68146cdbd18946f123693be with delete_type TABLET_DATA_TOMBSTONED (TS 14753c93125941acaad7283edc669052 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:53826
I20240502 14:00:38.847522 2164 tablet_replica.cc:330] T aaca0594b68146cdbd18946f123693be P 14753c93125941acaad7283edc669052: stopping tablet replica
I20240502 14:00:38.847651 2164 raft_consensus.cc:2229] T aaca0594b68146cdbd18946f123693be P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:38.847813 2164 raft_consensus.cc:2258] T aaca0594b68146cdbd18946f123693be P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:38.847854 30887 tablet_copy_source_session.cc:205] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:38.848554 2164 ts_tablet_manager.cc:1875] T aaca0594b68146cdbd18946f123693be P 14753c93125941acaad7283edc669052: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:38.848779 2717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a3c9c93437c43b1a9c0326f2880a04e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:38.850514 2164 ts_tablet_manager.cc:1888] T aaca0594b68146cdbd18946f123693be P 14753c93125941acaad7283edc669052: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:38.850569 2164 log.cc:1196] T aaca0594b68146cdbd18946f123693be P 14753c93125941acaad7283edc669052: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/aaca0594b68146cdbd18946f123693be
I20240502 14:00:38.850745 2717 tablet_copy_client.cc:783] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:38.850864 2717 tablet_copy_client.cc:647] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:38.851038 30701 catalog_manager.cc:4830] TS 14753c93125941acaad7283edc669052 (127.24.66.68:37029): tablet aaca0594b68146cdbd18946f123693be (table rebalance_test_table_2 [id=2109014949af4d09893b47c9b10ba7b5]) successfully deleted
I20240502 14:00:38.851850 2718 tablet_copy_client.cc:1002] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:38.857051 940 raft_consensus.cc:1050] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929: attempting to promote NON_VOTER f769c13ece0b40179ba120dbebbaba98 to VOTER
I20240502 14:00:38.857486 940 consensus_queue.cc:237] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } }
I20240502 14:00:38.858753 354 raft_consensus.cc:1261] T 5702766b82f24e96a4cb5da5f051fc3f P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.858753 30865 raft_consensus.cc:1261] T 5702766b82f24e96a4cb5da5f051fc3f P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.859162 31133 raft_consensus.cc:1261] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.859138 1538 consensus_queue.cc:1035] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.859302 2149 consensus_queue.cc:1035] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.859685 2149 consensus_queue.cc:1035] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.861496 1538 raft_consensus.cc:2932] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.861989 31133 raft_consensus.cc:2932] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.861968 354 raft_consensus.cc:2932] T 5702766b82f24e96a4cb5da5f051fc3f P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.863814 30865 raft_consensus.cc:2932] T 5702766b82f24e96a4cb5da5f051fc3f P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.864665 30707 catalog_manager.cc:5484] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:38.885749 2758 ts_tablet_manager.cc:902] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98: Initiating tablet copy from peer 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:38.886271 2758 tablet_copy_client.cc:308] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:37029
I20240502 14:00:38.889186 31288 tablet_copy_service.cc:140] P 14753c93125941acaad7283edc669052: Received BeginTabletCopySession request for tablet da2f3b3481da476d98c97bf35f055c2e from peer f769c13ece0b40179ba120dbebbaba98 ({username='slave'} at 127.24.66.70:38089)
I20240502 14:00:38.889282 31288 tablet_copy_service.cc:161] P 14753c93125941acaad7283edc669052: Beginning new tablet copy session on tablet da2f3b3481da476d98c97bf35f055c2e from peer f769c13ece0b40179ba120dbebbaba98 at {username='slave'} at 127.24.66.70:38089: session id = f769c13ece0b40179ba120dbebbaba98-da2f3b3481da476d98c97bf35f055c2e
I20240502 14:00:38.890100 31288 tablet_copy_source_session.cc:205] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:38.890832 2758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da2f3b3481da476d98c97bf35f055c2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:38.892107 31002 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5702766b82f24e96a4cb5da5f051fc3f"
dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:37246
I20240502 14:00:38.892246 31002 raft_consensus.cc:595] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:38.893155 2758 tablet_copy_client.cc:783] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:38.893291 2758 tablet_copy_client.cc:647] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:38.894244 2762 tablet_copy_client.cc:1002] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:38.899761 2314 raft_consensus.cc:1050] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052: attempting to promote NON_VOTER 3daf9e0837c74a919c89e686597ff56b to VOTER
I20240502 14:00:38.900148 2314 consensus_queue.cc:237] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:38.901821 30865 raft_consensus.cc:1261] T 2051076a81274b688554c218c107ee1c P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.901907 31002 raft_consensus.cc:1261] T 2051076a81274b688554c218c107ee1c P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.902200 2318 consensus_queue.cc:1035] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.902498 2318 consensus_queue.cc:1035] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.903141 488 raft_consensus.cc:1261] T 2051076a81274b688554c218c107ee1c P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:38.903499 1620 consensus_queue.cc:1035] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:38.904386 2318 raft_consensus.cc:2932] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.906041 488 raft_consensus.cc:2932] T 2051076a81274b688554c218c107ee1c P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.906852 30865 raft_consensus.cc:2932] T 2051076a81274b688554c218c107ee1c P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.907099 30713 catalog_manager.cc:5484] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:38.908798 31002 raft_consensus.cc:2932] T 2051076a81274b688554c218c107ee1c P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.910607 31266 consensus_queue.cc:237] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:38.912429 488 raft_consensus.cc:1261] T 2051076a81274b688554c218c107ee1c P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.912508 31002 raft_consensus.cc:1261] T 2051076a81274b688554c218c107ee1c P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:38.912812 1620 consensus_queue.cc:1035] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.913203 1620 consensus_queue.cc:1035] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:38.914748 2318 raft_consensus.cc:2932] T 2051076a81274b688554c218c107ee1c P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.915047 488 raft_consensus.cc:2932] T 2051076a81274b688554c218c107ee1c P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.916532 30701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2051076a81274b688554c218c107ee1c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:38.917078 30715 catalog_manager.cc:5484] T 2051076a81274b688554c218c107ee1c P 3daf9e0837c74a919c89e686597ff56b reported cstate change: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.917551 31002 raft_consensus.cc:2932] T 2051076a81274b688554c218c107ee1c P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:38.920179 30843 tablet_service.cc:1514] Processing DeleteTablet for tablet 2051076a81274b688554c218c107ee1c with delete_type TABLET_DATA_TOMBSTONED (TS ad5befacd63742a1b5b69fdacdb8cf8f not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45544
I20240502 14:00:38.920411 2304 tablet_replica.cc:330] T 2051076a81274b688554c218c107ee1c P ad5befacd63742a1b5b69fdacdb8cf8f: stopping tablet replica
I20240502 14:00:38.920542 2304 raft_consensus.cc:2229] T 2051076a81274b688554c218c107ee1c P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:38.920676 2304 raft_consensus.cc:2258] T 2051076a81274b688554c218c107ee1c P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:38.921356 2304 ts_tablet_manager.cc:1875] T 2051076a81274b688554c218c107ee1c P ad5befacd63742a1b5b69fdacdb8cf8f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:38.923169 2304 ts_tablet_manager.cc:1888] T 2051076a81274b688554c218c107ee1c P ad5befacd63742a1b5b69fdacdb8cf8f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:38.923235 2304 log.cc:1196] T 2051076a81274b688554c218c107ee1c P ad5befacd63742a1b5b69fdacdb8cf8f: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/wals/2051076a81274b688554c218c107ee1c
I20240502 14:00:38.923736 30698 catalog_manager.cc:4830] TS ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553): tablet 2051076a81274b688554c218c107ee1c (table rebalance_test_table_2 [id=2109014949af4d09893b47c9b10ba7b5]) successfully deleted
I20240502 14:00:38.923830 2792 ts_tablet_manager.cc:902] T 39239edeafb84b5caaa5dceea06fd4b0 P 3daf9e0837c74a919c89e686597ff56b: Initiating tablet copy from peer 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:38.924304 2792 tablet_copy_client.cc:308] T 39239edeafb84b5caaa5dceea06fd4b0 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:37029
I20240502 14:00:38.924710 31288 tablet_copy_service.cc:140] P 14753c93125941acaad7283edc669052: Received BeginTabletCopySession request for tablet 39239edeafb84b5caaa5dceea06fd4b0 from peer 3daf9e0837c74a919c89e686597ff56b ({username='slave'} at 127.24.66.71:44397)
I20240502 14:00:38.924795 31288 tablet_copy_service.cc:161] P 14753c93125941acaad7283edc669052: Beginning new tablet copy session on tablet 39239edeafb84b5caaa5dceea06fd4b0 from peer 3daf9e0837c74a919c89e686597ff56b at {username='slave'} at 127.24.66.71:44397: session id = 3daf9e0837c74a919c89e686597ff56b-39239edeafb84b5caaa5dceea06fd4b0
I20240502 14:00:38.925649 31288 tablet_copy_source_session.cc:205] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:38.926334 2792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39239edeafb84b5caaa5dceea06fd4b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:38.928262 2792 tablet_copy_client.cc:783] T 39239edeafb84b5caaa5dceea06fd4b0 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:38.928436 2792 tablet_copy_client.cc:647] T 39239edeafb84b5caaa5dceea06fd4b0 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:38.929351 2794 tablet_copy_client.cc:1002] T 39239edeafb84b5caaa5dceea06fd4b0 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:38.971130 31002 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5702766b82f24e96a4cb5da5f051fc3f"
dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:37278
I20240502 14:00:38.971230 31002 raft_consensus.cc:595] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:39.037000 2857 ts_tablet_manager.cc:902] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5: Initiating tablet copy from peer ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
I20240502 14:00:39.037449 2857 tablet_copy_client.cc:308] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.65:39553
I20240502 14:00:39.037861 30887 tablet_copy_service.cc:140] P ad5befacd63742a1b5b69fdacdb8cf8f: Received BeginTabletCopySession request for tablet efd8e3b804cf41bebf5e629b65571979 from peer 2716cee98b1b4c4591a40e1f466520e5 ({username='slave'} at 127.24.66.69:55949)
I20240502 14:00:39.037948 30887 tablet_copy_service.cc:161] P ad5befacd63742a1b5b69fdacdb8cf8f: Beginning new tablet copy session on tablet efd8e3b804cf41bebf5e629b65571979 from peer 2716cee98b1b4c4591a40e1f466520e5 at {username='slave'} at 127.24.66.69:55949: session id = 2716cee98b1b4c4591a40e1f466520e5-efd8e3b804cf41bebf5e629b65571979
I20240502 14:00:39.038607 30887 tablet_copy_source_session.cc:205] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:39.039351 2857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efd8e3b804cf41bebf5e629b65571979. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:39.041013 2857 tablet_copy_client.cc:783] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:39.041147 2857 tablet_copy_client.cc:647] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:39.042013 2861 tablet_copy_client.cc:1002] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:39.104028 2862 ts_tablet_manager.cc:902] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 2bb442cde32f426bb14791852dd81802: Initiating tablet copy from peer ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
I20240502 14:00:39.104537 2862 tablet_copy_client.cc:308] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 2bb442cde32f426bb14791852dd81802: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.65:39553
I20240502 14:00:39.104964 30887 tablet_copy_service.cc:140] P ad5befacd63742a1b5b69fdacdb8cf8f: Received BeginTabletCopySession request for tablet 4a12b8eaab7943c3bc9ee48c2a2307d0 from peer 2bb442cde32f426bb14791852dd81802 ({username='slave'} at 127.24.66.72:50901)
I20240502 14:00:39.105053 30887 tablet_copy_service.cc:161] P ad5befacd63742a1b5b69fdacdb8cf8f: Beginning new tablet copy session on tablet 4a12b8eaab7943c3bc9ee48c2a2307d0 from peer 2bb442cde32f426bb14791852dd81802 at {username='slave'} at 127.24.66.72:50901: session id = 2bb442cde32f426bb14791852dd81802-4a12b8eaab7943c3bc9ee48c2a2307d0
I20240502 14:00:39.105770 30887 tablet_copy_source_session.cc:205] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:39.106489 2862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a12b8eaab7943c3bc9ee48c2a2307d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:39.108016 2862 tablet_copy_client.cc:783] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 2bb442cde32f426bb14791852dd81802: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:39.108156 2862 tablet_copy_client.cc:647] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 2bb442cde32f426bb14791852dd81802: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:39.109000 2864 tablet_copy_client.cc:1002] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 2bb442cde32f426bb14791852dd81802: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:39.223429 2907 ts_tablet_manager.cc:902] T c0b9178646764824b17cbd842fbb0ff2 P f769c13ece0b40179ba120dbebbaba98: Initiating tablet copy from peer 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:39.223912 2907 tablet_copy_client.cc:308] T c0b9178646764824b17cbd842fbb0ff2 P f769c13ece0b40179ba120dbebbaba98: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.67:34193
I20240502 14:00:39.227325 31155 tablet_copy_service.cc:140] P 8c61042437444f5f92c76109dfb229fc: Received BeginTabletCopySession request for tablet c0b9178646764824b17cbd842fbb0ff2 from peer f769c13ece0b40179ba120dbebbaba98 ({username='slave'} at 127.24.66.70:53053)
I20240502 14:00:39.227428 31155 tablet_copy_service.cc:161] P 8c61042437444f5f92c76109dfb229fc: Beginning new tablet copy session on tablet c0b9178646764824b17cbd842fbb0ff2 from peer f769c13ece0b40179ba120dbebbaba98 at {username='slave'} at 127.24.66.70:53053: session id = f769c13ece0b40179ba120dbebbaba98-c0b9178646764824b17cbd842fbb0ff2
I20240502 14:00:39.228257 31155 tablet_copy_source_session.cc:205] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:39.229163 2907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0b9178646764824b17cbd842fbb0ff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:39.230927 2907 tablet_copy_client.cc:783] T c0b9178646764824b17cbd842fbb0ff2 P f769c13ece0b40179ba120dbebbaba98: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:39.231034 2907 tablet_copy_client.cc:647] T c0b9178646764824b17cbd842fbb0ff2 P f769c13ece0b40179ba120dbebbaba98: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:39.232045 2911 tablet_copy_client.cc:1002] T c0b9178646764824b17cbd842fbb0ff2 P f769c13ece0b40179ba120dbebbaba98: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:39.236410 31002 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5702766b82f24e96a4cb5da5f051fc3f"
dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:37296
I20240502 14:00:39.236533 31002 raft_consensus.cc:595] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:39.312052 2932 ts_tablet_manager.cc:902] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b: Initiating tablet copy from peer 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:39.312583 2932 tablet_copy_client.cc:308] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:37029
I20240502 14:00:39.313007 31288 tablet_copy_service.cc:140] P 14753c93125941acaad7283edc669052: Received BeginTabletCopySession request for tablet f571119d6f624fe88bf1effeaf466085 from peer 3daf9e0837c74a919c89e686597ff56b ({username='slave'} at 127.24.66.71:44397)
I20240502 14:00:39.313105 31288 tablet_copy_service.cc:161] P 14753c93125941acaad7283edc669052: Beginning new tablet copy session on tablet f571119d6f624fe88bf1effeaf466085 from peer 3daf9e0837c74a919c89e686597ff56b at {username='slave'} at 127.24.66.71:44397: session id = 3daf9e0837c74a919c89e686597ff56b-f571119d6f624fe88bf1effeaf466085
I20240502 14:00:39.313776 31288 tablet_copy_source_session.cc:205] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:39.314543 2932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f571119d6f624fe88bf1effeaf466085. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:39.316617 2932 tablet_copy_client.cc:783] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:39.316812 2932 tablet_copy_client.cc:647] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:39.317672 2934 tablet_copy_client.cc:1002] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:39.336676 2937 ts_tablet_manager.cc:902] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b: Initiating tablet copy from peer 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:39.337057 2937 tablet_copy_client.cc:308] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.66:36395
I20240502 14:00:39.337453 31023 tablet_copy_service.cc:140] P 1fab10dcfb8542fca2b965fed6c67929: Received BeginTabletCopySession request for tablet 2e02dc2a98d04bddafdd1b933d89b4dd from peer 3daf9e0837c74a919c89e686597ff56b ({username='slave'} at 127.24.66.71:34075)
I20240502 14:00:39.337560 31023 tablet_copy_service.cc:161] P 1fab10dcfb8542fca2b965fed6c67929: Beginning new tablet copy session on tablet 2e02dc2a98d04bddafdd1b933d89b4dd from peer 3daf9e0837c74a919c89e686597ff56b at {username='slave'} at 127.24.66.71:34075: session id = 3daf9e0837c74a919c89e686597ff56b-2e02dc2a98d04bddafdd1b933d89b4dd
I20240502 14:00:39.338165 31023 tablet_copy_source_session.cc:205] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:39.338843 2937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e02dc2a98d04bddafdd1b933d89b4dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:39.340287 2937 tablet_copy_client.cc:783] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:39.340451 2937 tablet_copy_client.cc:647] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:39.341288 2941 tablet_copy_client.cc:1002] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:39.351676 2942 ts_tablet_manager.cc:902] T 903f5d6c1faf4267b7f7d7dc32670a52 P 2bb442cde32f426bb14791852dd81802: Initiating tablet copy from peer ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
I20240502 14:00:39.352120 2942 tablet_copy_client.cc:308] T 903f5d6c1faf4267b7f7d7dc32670a52 P 2bb442cde32f426bb14791852dd81802: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.65:39553
I20240502 14:00:39.352596 30887 tablet_copy_service.cc:140] P ad5befacd63742a1b5b69fdacdb8cf8f: Received BeginTabletCopySession request for tablet 903f5d6c1faf4267b7f7d7dc32670a52 from peer 2bb442cde32f426bb14791852dd81802 ({username='slave'} at 127.24.66.72:50901)
I20240502 14:00:39.352702 30887 tablet_copy_service.cc:161] P ad5befacd63742a1b5b69fdacdb8cf8f: Beginning new tablet copy session on tablet 903f5d6c1faf4267b7f7d7dc32670a52 from peer 2bb442cde32f426bb14791852dd81802 at {username='slave'} at 127.24.66.72:50901: session id = 2bb442cde32f426bb14791852dd81802-903f5d6c1faf4267b7f7d7dc32670a52
I20240502 14:00:39.353358 30887 tablet_copy_source_session.cc:205] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:39.354087 2942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 903f5d6c1faf4267b7f7d7dc32670a52. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:39.355656 2942 tablet_copy_client.cc:783] T 903f5d6c1faf4267b7f7d7dc32670a52 P 2bb442cde32f426bb14791852dd81802: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:39.355826 2942 tablet_copy_client.cc:647] T 903f5d6c1faf4267b7f7d7dc32670a52 P 2bb442cde32f426bb14791852dd81802: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:39.356375 2149 raft_consensus.cc:979] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929: : Instructing follower 8c61042437444f5f92c76109dfb229fc to start an election
I20240502 14:00:39.356467 2149 raft_consensus.cc:1067] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Signalling peer 8c61042437444f5f92c76109dfb229fc to start an election
I20240502 14:00:39.356874 2946 tablet_copy_client.cc:1002] T 903f5d6c1faf4267b7f7d7dc32670a52 P 2bb442cde32f426bb14791852dd81802: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:39.356976 31133 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5702766b82f24e96a4cb5da5f051fc3f"
dest_uuid: "8c61042437444f5f92c76109dfb229fc"
from {username='slave'} at 127.24.66.66:57411
I20240502 14:00:39.357115 31133 raft_consensus.cc:482] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:00:39.357177 31133 raft_consensus.cc:3037] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:39.357888 31133 raft_consensus.cc:504] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } }
I20240502 14:00:39.358266 31133 leader_election.cc:288] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 2 election: Requested vote from peers ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553), 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395), f769c13ece0b40179ba120dbebbaba98 (127.24.66.70:38291)
I20240502 14:00:39.358505 30865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5702766b82f24e96a4cb5da5f051fc3f" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
I20240502 14:00:39.358516 31002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5702766b82f24e96a4cb5da5f051fc3f" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
I20240502 14:00:39.358628 30865 raft_consensus.cc:3037] T 5702766b82f24e96a4cb5da5f051fc3f P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:39.358628 31002 raft_consensus.cc:3032] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:00:39.358678 31002 raft_consensus.cc:729] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Running, Role: LEADER
I20240502 14:00:39.358721 354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5702766b82f24e96a4cb5da5f051fc3f" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "f769c13ece0b40179ba120dbebbaba98"
I20240502 14:00:39.358832 31002 consensus_queue.cc:260] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } }
I20240502 14:00:39.358880 354 raft_consensus.cc:3037] T 5702766b82f24e96a4cb5da5f051fc3f P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:39.358985 31002 raft_consensus.cc:3037] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:39.359568 30865 raft_consensus.cc:2454] T 5702766b82f24e96a4cb5da5f051fc3f P ad5befacd63742a1b5b69fdacdb8cf8f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 2.
I20240502 14:00:39.359587 354 raft_consensus.cc:2454] T 5702766b82f24e96a4cb5da5f051fc3f P f769c13ece0b40179ba120dbebbaba98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 2.
I20240502 14:00:39.359928 31002 raft_consensus.cc:2454] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 2.
I20240502 14:00:39.360190 31069 leader_election.cc:302] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1fab10dcfb8542fca2b965fed6c67929, 8c61042437444f5f92c76109dfb229fc, f769c13ece0b40179ba120dbebbaba98; no voters:
I20240502 14:00:39.360395 2285 raft_consensus.cc:2781] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:00:39.360510 2285 raft_consensus.cc:686] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [term 2 LEADER]: Becoming Leader. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Running, Role: LEADER
I20240502 14:00:39.360646 2285 consensus_queue.cc:237] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } }
I20240502 14:00:39.362331 30713 catalog_manager.cc:5484] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc reported cstate change: term changed from 1 to 2, leader changed from 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) to 8c61042437444f5f92c76109dfb229fc (127.24.66.67). New cstate: current_term: 2 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:39.750485 30865 raft_consensus.cc:1261] T 5702766b82f24e96a4cb5da5f051fc3f P ad5befacd63742a1b5b69fdacdb8cf8f [term 2 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 14:00:39.750869 2285 consensus_queue.cc:1035] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:39.869360 31002 raft_consensus.cc:1261] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [term 2 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 14:00:39.869724 3002 consensus_queue.cc:1035] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:39.872025 354 raft_consensus.cc:1261] T 5702766b82f24e96a4cb5da5f051fc3f P f769c13ece0b40179ba120dbebbaba98 [term 2 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 14:00:39.872845 3002 consensus_queue.cc:1035] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:40.205319 2589 tablet_copy_client.cc:523] T 66e2378d5c0f400299a170df8a939577 P 2bb442cde32f426bb14791852dd81802: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:40.206784 2589 tablet_bootstrap.cc:492] T 66e2378d5c0f400299a170df8a939577 P 2bb442cde32f426bb14791852dd81802: Bootstrap starting.
I20240502 14:00:40.223796 2589 tablet_bootstrap.cc:492] T 66e2378d5c0f400299a170df8a939577 P 2bb442cde32f426bb14791852dd81802: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:40.224077 2589 tablet_bootstrap.cc:492] T 66e2378d5c0f400299a170df8a939577 P 2bb442cde32f426bb14791852dd81802: Bootstrap complete.
I20240502 14:00:40.224256 2589 ts_tablet_manager.cc:1367] T 66e2378d5c0f400299a170df8a939577 P 2bb442cde32f426bb14791852dd81802: Time spent bootstrapping tablet: real 0.018s user 0.015s sys 0.000s
I20240502 14:00:40.224725 2589 raft_consensus.cc:348] T 66e2378d5c0f400299a170df8a939577 P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:40.224856 2589 raft_consensus.cc:729] T 66e2378d5c0f400299a170df8a939577 P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2bb442cde32f426bb14791852dd81802, State: Initialized, Role: LEARNER
I20240502 14:00:40.224987 2589 consensus_queue.cc:260] T 66e2378d5c0f400299a170df8a939577 P 2bb442cde32f426bb14791852dd81802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:40.225296 2589 ts_tablet_manager.cc:1398] T 66e2378d5c0f400299a170df8a939577 P 2bb442cde32f426bb14791852dd81802: Time spent starting tablet: real 0.001s user 0.005s sys 0.000s
I20240502 14:00:40.225741 31155 tablet_copy_service.cc:342] P 8c61042437444f5f92c76109dfb229fc: Request end of tablet copy session 2bb442cde32f426bb14791852dd81802-66e2378d5c0f400299a170df8a939577 received from {username='slave'} at 127.24.66.72:50377
I20240502 14:00:40.225823 31155 tablet_copy_service.cc:433] P 8c61042437444f5f92c76109dfb229fc: ending tablet copy session 2bb442cde32f426bb14791852dd81802-66e2378d5c0f400299a170df8a939577 on tablet 66e2378d5c0f400299a170df8a939577 with peer 2bb442cde32f426bb14791852dd81802
I20240502 14:00:40.295248 31133 consensus_queue.cc:237] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } }
I20240502 14:00:40.296264 30865 raft_consensus.cc:1261] T 5702766b82f24e96a4cb5da5f051fc3f P ad5befacd63742a1b5b69fdacdb8cf8f [term 2 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240502 14:00:40.296408 354 raft_consensus.cc:1261] T 5702766b82f24e96a4cb5da5f051fc3f P f769c13ece0b40179ba120dbebbaba98 [term 2 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240502 14:00:40.296602 32250 consensus_queue.cc:1035] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:40.296747 2285 consensus_queue.cc:1035] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:40.298239 3002 raft_consensus.cc:2932] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:40.298656 30865 raft_consensus.cc:2932] T 5702766b82f24e96a4cb5da5f051fc3f P ad5befacd63742a1b5b69fdacdb8cf8f [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:40.298698 354 raft_consensus.cc:2932] T 5702766b82f24e96a4cb5da5f051fc3f P f769c13ece0b40179ba120dbebbaba98 [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:40.300189 30701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5702766b82f24e96a4cb5da5f051fc3f with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:40.300925 30713 catalog_manager.cc:5484] T 5702766b82f24e96a4cb5da5f051fc3f P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index 5 to 7, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New cstate: current_term: 2 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:40.303143 30982 tablet_service.cc:1514] Processing DeleteTablet for tablet 5702766b82f24e96a4cb5da5f051fc3f with delete_type TABLET_DATA_TOMBSTONED (TS 1fab10dcfb8542fca2b965fed6c67929 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:58604
I20240502 14:00:40.303546 3181 tablet_replica.cc:330] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929: stopping tablet replica
I20240502 14:00:40.303674 3181 raft_consensus.cc:2229] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:40.303784 3181 raft_consensus.cc:2258] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:40.304409 3181 ts_tablet_manager.cc:1875] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:40.306005 3181 ts_tablet_manager.cc:1888] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240502 14:00:40.306072 3181 log.cc:1196] T 5702766b82f24e96a4cb5da5f051fc3f P 1fab10dcfb8542fca2b965fed6c67929: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/wals/5702766b82f24e96a4cb5da5f051fc3f
I20240502 14:00:40.306545 30700 catalog_manager.cc:4830] TS 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395): tablet 5702766b82f24e96a4cb5da5f051fc3f (table rebalance_test_table_3 [id=3fb9e83c8b7c4d90a375d14e792f650d]) successfully deleted
I20240502 14:00:40.352092 2717 tablet_copy_client.cc:523] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:40.353384 2717 tablet_bootstrap.cc:492] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98: Bootstrap starting.
I20240502 14:00:40.369340 2717 tablet_bootstrap.cc:492] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:40.369581 2717 tablet_bootstrap.cc:492] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98: Bootstrap complete.
I20240502 14:00:40.369724 2717 ts_tablet_manager.cc:1367] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98: Time spent bootstrapping tablet: real 0.016s user 0.016s sys 0.000s
I20240502 14:00:40.370054 2717 raft_consensus.cc:348] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:40.370147 2717 raft_consensus.cc:729] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f769c13ece0b40179ba120dbebbaba98, State: Initialized, Role: LEARNER
I20240502 14:00:40.370236 2717 consensus_queue.cc:260] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:40.370468 2717 ts_tablet_manager.cc:1398] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:40.370910 30887 tablet_copy_service.cc:342] P ad5befacd63742a1b5b69fdacdb8cf8f: Request end of tablet copy session f769c13ece0b40179ba120dbebbaba98-5a3c9c93437c43b1a9c0326f2880a04e received from {username='slave'} at 127.24.66.70:52845
I20240502 14:00:40.370996 30887 tablet_copy_service.cc:433] P ad5befacd63742a1b5b69fdacdb8cf8f: ending tablet copy session f769c13ece0b40179ba120dbebbaba98-5a3c9c93437c43b1a9c0326f2880a04e on tablet 5a3c9c93437c43b1a9c0326f2880a04e with peer f769c13ece0b40179ba120dbebbaba98
I20240502 14:00:40.394479 2758 tablet_copy_client.cc:523] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:40.395604 2758 tablet_bootstrap.cc:492] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98: Bootstrap starting.
I20240502 14:00:40.412582 2758 tablet_bootstrap.cc:492] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:40.412812 2758 tablet_bootstrap.cc:492] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98: Bootstrap complete.
I20240502 14:00:40.412961 2758 ts_tablet_manager.cc:1367] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98: Time spent bootstrapping tablet: real 0.017s user 0.017s sys 0.000s
I20240502 14:00:40.413244 2758 raft_consensus.cc:348] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:40.413329 2758 raft_consensus.cc:729] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f769c13ece0b40179ba120dbebbaba98, State: Initialized, Role: LEARNER
I20240502 14:00:40.413442 2758 consensus_queue.cc:260] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:40.413677 2758 ts_tablet_manager.cc:1398] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:40.414153 31288 tablet_copy_service.cc:342] P 14753c93125941acaad7283edc669052: Request end of tablet copy session f769c13ece0b40179ba120dbebbaba98-da2f3b3481da476d98c97bf35f055c2e received from {username='slave'} at 127.24.66.70:38089
I20240502 14:00:40.414237 31288 tablet_copy_service.cc:433] P 14753c93125941acaad7283edc669052: ending tablet copy session f769c13ece0b40179ba120dbebbaba98-da2f3b3481da476d98c97bf35f055c2e on tablet da2f3b3481da476d98c97bf35f055c2e with peer f769c13ece0b40179ba120dbebbaba98
I20240502 14:00:40.428802 354 raft_consensus.cc:1203] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:40.429641 2792 tablet_copy_client.cc:523] T 39239edeafb84b5caaa5dceea06fd4b0 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:40.430795 2792 tablet_bootstrap.cc:492] T 39239edeafb84b5caaa5dceea06fd4b0 P 3daf9e0837c74a919c89e686597ff56b: Bootstrap starting.
I20240502 14:00:40.446156 2792 tablet_bootstrap.cc:492] T 39239edeafb84b5caaa5dceea06fd4b0 P 3daf9e0837c74a919c89e686597ff56b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:40.446413 2792 tablet_bootstrap.cc:492] T 39239edeafb84b5caaa5dceea06fd4b0 P 3daf9e0837c74a919c89e686597ff56b: Bootstrap complete.
I20240502 14:00:40.446563 2792 ts_tablet_manager.cc:1367] T 39239edeafb84b5caaa5dceea06fd4b0 P 3daf9e0837c74a919c89e686597ff56b: Time spent bootstrapping tablet: real 0.016s user 0.013s sys 0.001s
I20240502 14:00:40.446967 2792 raft_consensus.cc:348] T 39239edeafb84b5caaa5dceea06fd4b0 P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:40.447063 2792 raft_consensus.cc:729] T 39239edeafb84b5caaa5dceea06fd4b0 P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3daf9e0837c74a919c89e686597ff56b, State: Initialized, Role: LEARNER
I20240502 14:00:40.447166 2792 consensus_queue.cc:260] T 39239edeafb84b5caaa5dceea06fd4b0 P 3daf9e0837c74a919c89e686597ff56b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:40.447463 2792 ts_tablet_manager.cc:1398] T 39239edeafb84b5caaa5dceea06fd4b0 P 3daf9e0837c74a919c89e686597ff56b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:00:40.447937 31288 tablet_copy_service.cc:342] P 14753c93125941acaad7283edc669052: Request end of tablet copy session 3daf9e0837c74a919c89e686597ff56b-39239edeafb84b5caaa5dceea06fd4b0 received from {username='slave'} at 127.24.66.71:44397
I20240502 14:00:40.448021 31288 tablet_copy_service.cc:433] P 14753c93125941acaad7283edc669052: ending tablet copy session 3daf9e0837c74a919c89e686597ff56b-39239edeafb84b5caaa5dceea06fd4b0 on tablet 39239edeafb84b5caaa5dceea06fd4b0 with peer 3daf9e0837c74a919c89e686597ff56b
I20240502 14:00:40.450917 487 raft_consensus.cc:1203] T 39239edeafb84b5caaa5dceea06fd4b0 P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:40.497480 619 raft_consensus.cc:1203] T 66e2378d5c0f400299a170df8a939577 P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:40.542526 2857 tablet_copy_client.cc:523] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:40.544018 2857 tablet_bootstrap.cc:492] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap starting.
I20240502 14:00:40.564888 2857 tablet_bootstrap.cc:492] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:40.565155 2857 tablet_bootstrap.cc:492] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap complete.
I20240502 14:00:40.565325 2857 ts_tablet_manager.cc:1367] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5: Time spent bootstrapping tablet: real 0.021s user 0.013s sys 0.007s
I20240502 14:00:40.565681 2857 raft_consensus.cc:348] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:40.565876 2857 raft_consensus.cc:729] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2716cee98b1b4c4591a40e1f466520e5, State: Initialized, Role: LEARNER
I20240502 14:00:40.565994 2857 consensus_queue.cc:260] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:40.566331 2857 ts_tablet_manager.cc:1398] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:40.566881 30887 tablet_copy_service.cc:342] P ad5befacd63742a1b5b69fdacdb8cf8f: Request end of tablet copy session 2716cee98b1b4c4591a40e1f466520e5-efd8e3b804cf41bebf5e629b65571979 received from {username='slave'} at 127.24.66.69:55949
I20240502 14:00:40.566947 30887 tablet_copy_service.cc:433] P ad5befacd63742a1b5b69fdacdb8cf8f: ending tablet copy session 2716cee98b1b4c4591a40e1f466520e5-efd8e3b804cf41bebf5e629b65571979 on tablet efd8e3b804cf41bebf5e629b65571979 with peer 2716cee98b1b4c4591a40e1f466520e5
I20240502 14:00:40.569924 32691 raft_consensus.cc:1203] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:40.609247 2862 tablet_copy_client.cc:523] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 2bb442cde32f426bb14791852dd81802: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:40.610405 2862 tablet_bootstrap.cc:492] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 2bb442cde32f426bb14791852dd81802: Bootstrap starting.
I20240502 14:00:40.628046 2862 tablet_bootstrap.cc:492] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 2bb442cde32f426bb14791852dd81802: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:40.628274 2862 tablet_bootstrap.cc:492] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 2bb442cde32f426bb14791852dd81802: Bootstrap complete.
I20240502 14:00:40.628465 2862 ts_tablet_manager.cc:1367] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 2bb442cde32f426bb14791852dd81802: Time spent bootstrapping tablet: real 0.018s user 0.017s sys 0.002s
I20240502 14:00:40.628878 2862 raft_consensus.cc:348] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:40.628975 2862 raft_consensus.cc:729] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2bb442cde32f426bb14791852dd81802, State: Initialized, Role: LEARNER
I20240502 14:00:40.629094 2862 consensus_queue.cc:260] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 2bb442cde32f426bb14791852dd81802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:40.629338 2862 ts_tablet_manager.cc:1398] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 2bb442cde32f426bb14791852dd81802: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:40.629796 30887 tablet_copy_service.cc:342] P ad5befacd63742a1b5b69fdacdb8cf8f: Request end of tablet copy session 2bb442cde32f426bb14791852dd81802-4a12b8eaab7943c3bc9ee48c2a2307d0 received from {username='slave'} at 127.24.66.72:50901
I20240502 14:00:40.629881 30887 tablet_copy_service.cc:433] P ad5befacd63742a1b5b69fdacdb8cf8f: ending tablet copy session 2bb442cde32f426bb14791852dd81802-4a12b8eaab7943c3bc9ee48c2a2307d0 on tablet 4a12b8eaab7943c3bc9ee48c2a2307d0 with peer 2bb442cde32f426bb14791852dd81802
I20240502 14:00:40.732422 2907 tablet_copy_client.cc:523] T c0b9178646764824b17cbd842fbb0ff2 P f769c13ece0b40179ba120dbebbaba98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:40.736083 2907 tablet_bootstrap.cc:492] T c0b9178646764824b17cbd842fbb0ff2 P f769c13ece0b40179ba120dbebbaba98: Bootstrap starting.
I20240502 14:00:40.752473 2907 tablet_bootstrap.cc:492] T c0b9178646764824b17cbd842fbb0ff2 P f769c13ece0b40179ba120dbebbaba98: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:40.752863 2907 tablet_bootstrap.cc:492] T c0b9178646764824b17cbd842fbb0ff2 P f769c13ece0b40179ba120dbebbaba98: Bootstrap complete.
I20240502 14:00:40.753015 2907 ts_tablet_manager.cc:1367] T c0b9178646764824b17cbd842fbb0ff2 P f769c13ece0b40179ba120dbebbaba98: Time spent bootstrapping tablet: real 0.017s user 0.017s sys 0.000s
I20240502 14:00:40.753376 2907 raft_consensus.cc:348] T c0b9178646764824b17cbd842fbb0ff2 P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:40.753480 2907 raft_consensus.cc:729] T c0b9178646764824b17cbd842fbb0ff2 P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f769c13ece0b40179ba120dbebbaba98, State: Initialized, Role: LEARNER
I20240502 14:00:40.753576 2907 consensus_queue.cc:260] T c0b9178646764824b17cbd842fbb0ff2 P f769c13ece0b40179ba120dbebbaba98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:40.753790 2907 ts_tablet_manager.cc:1398] T c0b9178646764824b17cbd842fbb0ff2 P f769c13ece0b40179ba120dbebbaba98: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:40.754241 31155 tablet_copy_service.cc:342] P 8c61042437444f5f92c76109dfb229fc: Request end of tablet copy session f769c13ece0b40179ba120dbebbaba98-c0b9178646764824b17cbd842fbb0ff2 received from {username='slave'} at 127.24.66.70:53053
I20240502 14:00:40.754324 31155 tablet_copy_service.cc:433] P 8c61042437444f5f92c76109dfb229fc: ending tablet copy session f769c13ece0b40179ba120dbebbaba98-c0b9178646764824b17cbd842fbb0ff2 on tablet c0b9178646764824b17cbd842fbb0ff2 with peer f769c13ece0b40179ba120dbebbaba98
I20240502 14:00:40.817951 2932 tablet_copy_client.cc:523] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:40.819432 2932 tablet_bootstrap.cc:492] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b: Bootstrap starting.
I20240502 14:00:40.840914 2932 tablet_bootstrap.cc:492] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:40.841194 2932 tablet_bootstrap.cc:492] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b: Bootstrap complete.
I20240502 14:00:40.841367 2932 ts_tablet_manager.cc:1367] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b: Time spent bootstrapping tablet: real 0.022s user 0.017s sys 0.004s
I20240502 14:00:40.841552 2937 tablet_copy_client.cc:523] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:40.841723 2932 raft_consensus.cc:348] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:40.841820 2932 raft_consensus.cc:729] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3daf9e0837c74a919c89e686597ff56b, State: Initialized, Role: LEARNER
I20240502 14:00:40.841887 2932 consensus_queue.cc:260] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:40.842134 2932 ts_tablet_manager.cc:1398] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:40.842602 31288 tablet_copy_service.cc:342] P 14753c93125941acaad7283edc669052: Request end of tablet copy session 3daf9e0837c74a919c89e686597ff56b-f571119d6f624fe88bf1effeaf466085 received from {username='slave'} at 127.24.66.71:44397
I20240502 14:00:40.842675 31288 tablet_copy_service.cc:433] P 14753c93125941acaad7283edc669052: ending tablet copy session 3daf9e0837c74a919c89e686597ff56b-f571119d6f624fe88bf1effeaf466085 on tablet f571119d6f624fe88bf1effeaf466085 with peer 3daf9e0837c74a919c89e686597ff56b
I20240502 14:00:40.842844 2937 tablet_bootstrap.cc:492] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b: Bootstrap starting.
I20240502 14:00:40.857141 2942 tablet_copy_client.cc:523] T 903f5d6c1faf4267b7f7d7dc32670a52 P 2bb442cde32f426bb14791852dd81802: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:40.858282 2942 tablet_bootstrap.cc:492] T 903f5d6c1faf4267b7f7d7dc32670a52 P 2bb442cde32f426bb14791852dd81802: Bootstrap starting.
I20240502 14:00:40.859678 2937 tablet_bootstrap.cc:492] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:40.859913 2937 tablet_bootstrap.cc:492] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b: Bootstrap complete.
I20240502 14:00:40.860071 2937 ts_tablet_manager.cc:1367] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b: Time spent bootstrapping tablet: real 0.017s user 0.013s sys 0.001s
I20240502 14:00:40.860548 2937 raft_consensus.cc:348] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:40.860673 2937 raft_consensus.cc:729] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3daf9e0837c74a919c89e686597ff56b, State: Initialized, Role: LEARNER
I20240502 14:00:40.860778 2937 consensus_queue.cc:260] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:40.861085 2937 ts_tablet_manager.cc:1398] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 14:00:40.861549 31023 tablet_copy_service.cc:342] P 1fab10dcfb8542fca2b965fed6c67929: Request end of tablet copy session 3daf9e0837c74a919c89e686597ff56b-2e02dc2a98d04bddafdd1b933d89b4dd received from {username='slave'} at 127.24.66.71:34075
I20240502 14:00:40.861636 31023 tablet_copy_service.cc:433] P 1fab10dcfb8542fca2b965fed6c67929: ending tablet copy session 3daf9e0837c74a919c89e686597ff56b-2e02dc2a98d04bddafdd1b933d89b4dd on tablet 2e02dc2a98d04bddafdd1b933d89b4dd with peer 3daf9e0837c74a919c89e686597ff56b
I20240502 14:00:40.876327 2942 tablet_bootstrap.cc:492] T 903f5d6c1faf4267b7f7d7dc32670a52 P 2bb442cde32f426bb14791852dd81802: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:40.876567 2942 tablet_bootstrap.cc:492] T 903f5d6c1faf4267b7f7d7dc32670a52 P 2bb442cde32f426bb14791852dd81802: Bootstrap complete.
I20240502 14:00:40.876690 2942 ts_tablet_manager.cc:1367] T 903f5d6c1faf4267b7f7d7dc32670a52 P 2bb442cde32f426bb14791852dd81802: Time spent bootstrapping tablet: real 0.018s user 0.018s sys 0.000s
I20240502 14:00:40.876984 2942 raft_consensus.cc:348] T 903f5d6c1faf4267b7f7d7dc32670a52 P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:40.877075 2942 raft_consensus.cc:729] T 903f5d6c1faf4267b7f7d7dc32670a52 P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2bb442cde32f426bb14791852dd81802, State: Initialized, Role: LEARNER
I20240502 14:00:40.877146 2942 consensus_queue.cc:260] T 903f5d6c1faf4267b7f7d7dc32670a52 P 2bb442cde32f426bb14791852dd81802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:40.877410 2942 ts_tablet_manager.cc:1398] T 903f5d6c1faf4267b7f7d7dc32670a52 P 2bb442cde32f426bb14791852dd81802: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:40.877877 30887 tablet_copy_service.cc:342] P ad5befacd63742a1b5b69fdacdb8cf8f: Request end of tablet copy session 2bb442cde32f426bb14791852dd81802-903f5d6c1faf4267b7f7d7dc32670a52 received from {username='slave'} at 127.24.66.72:50901
I20240502 14:00:40.877964 30887 tablet_copy_service.cc:433] P ad5befacd63742a1b5b69fdacdb8cf8f: ending tablet copy session 2bb442cde32f426bb14791852dd81802-903f5d6c1faf4267b7f7d7dc32670a52 on tablet 903f5d6c1faf4267b7f7d7dc32670a52 with peer 2bb442cde32f426bb14791852dd81802
I20240502 14:00:40.881757 487 raft_consensus.cc:1203] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:40.919688 3350 raft_consensus.cc:1050] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f: attempting to promote NON_VOTER f769c13ece0b40179ba120dbebbaba98 to VOTER
I20240502 14:00:40.920111 3350 consensus_queue.cc:237] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } }
I20240502 14:00:40.921386 31266 raft_consensus.cc:1261] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:40.921393 354 raft_consensus.cc:1261] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:40.921489 31133 raft_consensus.cc:1261] T 5a3c9c93437c43b1a9c0326f2880a04e P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:40.921754 3353 consensus_queue.cc:1035] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:40.921933 3350 consensus_queue.cc:1035] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:40.922098 3352 consensus_queue.cc:1035] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:40.923846 3350 raft_consensus.cc:2932] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:40.924364 354 raft_consensus.cc:2932] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:40.924427 31266 raft_consensus.cc:2932] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:40.926780 31133 raft_consensus.cc:2932] T 5a3c9c93437c43b1a9c0326f2880a04e P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:40.926826 30715 catalog_manager.cc:5484] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:40.930516 30865 consensus_queue.cc:237] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } }
I20240502 14:00:40.931411 354 raft_consensus.cc:1261] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:40.931514 31266 raft_consensus.cc:1261] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:40.931761 3350 consensus_queue.cc:1035] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:40.931931 3352 consensus_queue.cc:1035] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:40.933440 2183 raft_consensus.cc:2932] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:40.933743 31266 raft_consensus.cc:2932] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:40.935170 354 raft_consensus.cc:2932] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:40.935863 30710 catalog_manager.cc:5484] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:40.936383 30698 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5a3c9c93437c43b1a9c0326f2880a04e with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:40.937990 31115 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a3c9c93437c43b1a9c0326f2880a04e with delete_type TABLET_DATA_TOMBSTONED (TS 8c61042437444f5f92c76109dfb229fc not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:42546
I20240502 14:00:40.938377 3360 tablet_replica.cc:330] T 5a3c9c93437c43b1a9c0326f2880a04e P 8c61042437444f5f92c76109dfb229fc: stopping tablet replica
I20240502 14:00:40.938530 3360 raft_consensus.cc:2229] T 5a3c9c93437c43b1a9c0326f2880a04e P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:40.938724 3360 raft_consensus.cc:2258] T 5a3c9c93437c43b1a9c0326f2880a04e P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:40.939267 3360 ts_tablet_manager.cc:1875] T 5a3c9c93437c43b1a9c0326f2880a04e P 8c61042437444f5f92c76109dfb229fc: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:40.941167 3360 ts_tablet_manager.cc:1888] T 5a3c9c93437c43b1a9c0326f2880a04e P 8c61042437444f5f92c76109dfb229fc: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:40.941228 3360 log.cc:1196] T 5a3c9c93437c43b1a9c0326f2880a04e P 8c61042437444f5f92c76109dfb229fc: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/wals/5a3c9c93437c43b1a9c0326f2880a04e
I20240502 14:00:40.941605 30701 catalog_manager.cc:4830] TS 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193): tablet 5a3c9c93437c43b1a9c0326f2880a04e (table rebalance_test_table_8 [id=5cb834cd98904017a28fc7b595efc564]) successfully deleted
I20240502 14:00:40.947947 354 raft_consensus.cc:1203] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:40.960675 3362 raft_consensus.cc:1050] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052: attempting to promote NON_VOTER 3daf9e0837c74a919c89e686597ff56b to VOTER
I20240502 14:00:40.961079 3362 consensus_queue.cc:237] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:40.962256 31133 raft_consensus.cc:1261] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:40.962378 31002 raft_consensus.cc:1261] T 39239edeafb84b5caaa5dceea06fd4b0 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:40.962502 487 raft_consensus.cc:1261] T 39239edeafb84b5caaa5dceea06fd4b0 P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:40.962795 3364 consensus_queue.cc:1035] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:40.962941 3362 consensus_queue.cc:1035] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:40.963052 3361 consensus_queue.cc:1035] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:40.965065 3361 raft_consensus.cc:2932] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:40.965654 31001 raft_consensus.cc:2932] T 39239edeafb84b5caaa5dceea06fd4b0 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:40.967306 31133 raft_consensus.cc:2932] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:40.967657 30713 catalog_manager.cc:5484] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:40.969210 487 raft_consensus.cc:2932] T 39239edeafb84b5caaa5dceea06fd4b0 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:40.971452 31266 consensus_queue.cc:237] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:40.972431 488 raft_consensus.cc:1261] T 39239edeafb84b5caaa5dceea06fd4b0 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:40.972424 31133 raft_consensus.cc:1261] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:40.972740 3362 consensus_queue.cc:1035] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:40.973030 3362 consensus_queue.cc:1035] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:40.974293 3362 raft_consensus.cc:2932] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:40.974687 488 raft_consensus.cc:2932] T 39239edeafb84b5caaa5dceea06fd4b0 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:40.975692 30701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 39239edeafb84b5caaa5dceea06fd4b0 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:40.976608 31133 raft_consensus.cc:2932] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:40.976703 30713 catalog_manager.cc:5484] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index 5 to 6, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:40.979322 30982 tablet_service.cc:1514] Processing DeleteTablet for tablet 39239edeafb84b5caaa5dceea06fd4b0 with delete_type TABLET_DATA_TOMBSTONED (TS 1fab10dcfb8542fca2b965fed6c67929 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:58604
I20240502 14:00:40.979620 3368 tablet_replica.cc:330] T 39239edeafb84b5caaa5dceea06fd4b0 P 1fab10dcfb8542fca2b965fed6c67929: stopping tablet replica
I20240502 14:00:40.979771 3368 raft_consensus.cc:2229] T 39239edeafb84b5caaa5dceea06fd4b0 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:40.979914 3368 raft_consensus.cc:2258] T 39239edeafb84b5caaa5dceea06fd4b0 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:40.981026 3368 ts_tablet_manager.cc:1875] T 39239edeafb84b5caaa5dceea06fd4b0 P 1fab10dcfb8542fca2b965fed6c67929: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:40.982627 3368 ts_tablet_manager.cc:1888] T 39239edeafb84b5caaa5dceea06fd4b0 P 1fab10dcfb8542fca2b965fed6c67929: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:40.982692 3368 log.cc:1196] T 39239edeafb84b5caaa5dceea06fd4b0 P 1fab10dcfb8542fca2b965fed6c67929: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/wals/39239edeafb84b5caaa5dceea06fd4b0
I20240502 14:00:40.983084 30700 catalog_manager.cc:4830] TS 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395): tablet 39239edeafb84b5caaa5dceea06fd4b0 (table rebalance_test_table_5 [id=b000c5e9f8984019bef07f52b5bff49d]) successfully deleted
I20240502 14:00:41.018086 354 raft_consensus.cc:1203] T c0b9178646764824b17cbd842fbb0ff2 P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:41.078222 619 raft_consensus.cc:1203] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:41.083840 3413 raft_consensus.cc:1050] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc: attempting to promote NON_VOTER 2bb442cde32f426bb14791852dd81802 to VOTER
I20240502 14:00:41.084228 3413 consensus_queue.cc:237] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } }
I20240502 14:00:41.085491 619 raft_consensus.cc:1261] T 66e2378d5c0f400299a170df8a939577 P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:41.085753 30865 raft_consensus.cc:1261] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:41.086050 31266 raft_consensus.cc:1261] T 66e2378d5c0f400299a170df8a939577 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:41.086084 3413 consensus_queue.cc:1035] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:41.086532 3413 consensus_queue.cc:1035] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:41.086666 32250 consensus_queue.cc:1035] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:41.088585 32250 raft_consensus.cc:2932] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:41.089023 31266 raft_consensus.cc:2932] T 66e2378d5c0f400299a170df8a939577 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:41.090270 619 raft_consensus.cc:2932] T 66e2378d5c0f400299a170df8a939577 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:41.090255 30865 raft_consensus.cc:2932] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:41.090888 30707 catalog_manager.cc:5484] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:41.094527 31133 consensus_queue.cc:237] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } }
I20240502 14:00:41.095584 30865 raft_consensus.cc:1261] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:41.095688 619 raft_consensus.cc:1261] T 66e2378d5c0f400299a170df8a939577 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:41.096098 3413 consensus_queue.cc:1035] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:41.096267 3349 consensus_queue.cc:1035] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:41.097586 32250 raft_consensus.cc:2932] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:41.098034 30865 raft_consensus.cc:2932] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:41.098786 30701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 66e2378d5c0f400299a170df8a939577 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:41.099567 619 raft_consensus.cc:2932] T 66e2378d5c0f400299a170df8a939577 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:41.099964 30713 catalog_manager.cc:5484] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:41.102260 31246 tablet_service.cc:1514] Processing DeleteTablet for tablet 66e2378d5c0f400299a170df8a939577 with delete_type TABLET_DATA_TOMBSTONED (TS 14753c93125941acaad7283edc669052 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:53826
I20240502 14:00:41.108503 3424 tablet_replica.cc:330] T 66e2378d5c0f400299a170df8a939577 P 14753c93125941acaad7283edc669052: stopping tablet replica
I20240502 14:00:41.108644 3424 raft_consensus.cc:2229] T 66e2378d5c0f400299a170df8a939577 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:41.108790 3424 raft_consensus.cc:2258] T 66e2378d5c0f400299a170df8a939577 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:41.109387 3424 ts_tablet_manager.cc:1875] T 66e2378d5c0f400299a170df8a939577 P 14753c93125941acaad7283edc669052: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:41.111518 3424 ts_tablet_manager.cc:1888] T 66e2378d5c0f400299a170df8a939577 P 14753c93125941acaad7283edc669052: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:41.111596 3424 log.cc:1196] T 66e2378d5c0f400299a170df8a939577 P 14753c93125941acaad7283edc669052: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/66e2378d5c0f400299a170df8a939577
I20240502 14:00:41.112128 30701 catalog_manager.cc:4830] TS 14753c93125941acaad7283edc669052 (127.24.66.68:37029): tablet 66e2378d5c0f400299a170df8a939577 (table rebalance_test_table_4 [id=4cd0d0b0efb44a2c966dd894b6e48601]) successfully deleted
I20240502 14:00:41.125412 3353 raft_consensus.cc:1050] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f: attempting to promote NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 to VOTER
I20240502 14:00:41.125756 3353 consensus_queue.cc:237] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } }
I20240502 14:00:41.127110 31133 raft_consensus.cc:1261] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:41.127203 31001 raft_consensus.cc:1261] T efd8e3b804cf41bebf5e629b65571979 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:41.127468 3352 consensus_queue.cc:1035] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:41.127751 2183 consensus_queue.cc:1035] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:41.127880 32691 raft_consensus.cc:1261] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:41.128232 2183 consensus_queue.cc:1035] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:41.129828 3352 raft_consensus.cc:2932] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:41.130897 31002 raft_consensus.cc:2932] T efd8e3b804cf41bebf5e629b65571979 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:41.131630 31133 raft_consensus.cc:2932] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:41.131848 32691 raft_consensus.cc:2932] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:41.133332 30713 catalog_manager.cc:5484] T efd8e3b804cf41bebf5e629b65571979 P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index 4 to 5, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:41.137392 30865 consensus_queue.cc:237] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } }
I20240502 14:00:41.138929 32691 raft_consensus.cc:1261] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:41.139298 2183 consensus_queue.cc:1035] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:41.139781 31133 raft_consensus.cc:1261] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:41.140084 2183 consensus_queue.cc:1035] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:41.141126 2183 raft_consensus.cc:2932] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:41.142271 30698 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet efd8e3b804cf41bebf5e629b65571979 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:41.142695 32691 raft_consensus.cc:2932] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:41.143214 488 raft_consensus.cc:1203] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:41.143630 30715 catalog_manager.cc:5484] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index 5 to 6, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:41.145011 31133 raft_consensus.cc:2932] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:41.146389 30982 tablet_service.cc:1514] Processing DeleteTablet for tablet efd8e3b804cf41bebf5e629b65571979 with delete_type TABLET_DATA_TOMBSTONED (TS 1fab10dcfb8542fca2b965fed6c67929 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:58604
I20240502 14:00:41.146616 3368 tablet_replica.cc:330] T efd8e3b804cf41bebf5e629b65571979 P 1fab10dcfb8542fca2b965fed6c67929: stopping tablet replica
I20240502 14:00:41.146766 3368 raft_consensus.cc:2229] T efd8e3b804cf41bebf5e629b65571979 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:41.146929 3368 raft_consensus.cc:2258] T efd8e3b804cf41bebf5e629b65571979 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:41.147987 3368 ts_tablet_manager.cc:1875] T efd8e3b804cf41bebf5e629b65571979 P 1fab10dcfb8542fca2b965fed6c67929: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:41.149994 3368 ts_tablet_manager.cc:1888] T efd8e3b804cf41bebf5e629b65571979 P 1fab10dcfb8542fca2b965fed6c67929: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:41.150053 3368 log.cc:1196] T efd8e3b804cf41bebf5e629b65571979 P 1fab10dcfb8542fca2b965fed6c67929: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/wals/efd8e3b804cf41bebf5e629b65571979
I20240502 14:00:41.150442 30700 catalog_manager.cc:4830] TS 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395): tablet efd8e3b804cf41bebf5e629b65571979 (table rebalance_test_table_4 [id=4cd0d0b0efb44a2c966dd894b6e48601]) successfully deleted
I20240502 14:00:41.347606 1620 raft_consensus.cc:1050] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052: attempting to promote NON_VOTER f769c13ece0b40179ba120dbebbaba98 to VOTER
I20240502 14:00:41.348119 1620 consensus_queue.cc:237] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } }
I20240502 14:00:41.349512 354 raft_consensus.cc:1261] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:41.349776 31133 raft_consensus.cc:1261] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:41.350157 3364 consensus_queue.cc:1035] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:41.350199 30865 raft_consensus.cc:1261] T da2f3b3481da476d98c97bf35f055c2e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:41.350317 3362 consensus_queue.cc:1035] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:41.350744 3362 consensus_queue.cc:1035] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:41.352674 3361 raft_consensus.cc:2932] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:41.353399 31133 raft_consensus.cc:2932] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:41.354887 349 raft_consensus.cc:2932] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:41.355480 30865 raft_consensus.cc:2932] T da2f3b3481da476d98c97bf35f055c2e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:41.355844 619 raft_consensus.cc:1203] T 903f5d6c1faf4267b7f7d7dc32670a52 P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:00:41.357923 30715 catalog_manager.cc:5484] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:41.361557 31266 consensus_queue.cc:237] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } }
I20240502 14:00:41.362896 349 raft_consensus.cc:1261] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:41.362885 31133 raft_consensus.cc:1261] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:41.363229 3362 consensus_queue.cc:1035] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:41.363399 3364 consensus_queue.cc:1035] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:41.365129 3362 raft_consensus.cc:2932] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:41.365583 349 raft_consensus.cc:2932] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:41.366212 31133 raft_consensus.cc:2932] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:41.367138 30701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet da2f3b3481da476d98c97bf35f055c2e with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:41.367882 30713 catalog_manager.cc:5484] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98 reported cstate change: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:41.370747 30843 tablet_service.cc:1514] Processing DeleteTablet for tablet da2f3b3481da476d98c97bf35f055c2e with delete_type TABLET_DATA_TOMBSTONED (TS ad5befacd63742a1b5b69fdacdb8cf8f not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45544
I20240502 14:00:41.371138 3539 tablet_replica.cc:330] T da2f3b3481da476d98c97bf35f055c2e P ad5befacd63742a1b5b69fdacdb8cf8f: stopping tablet replica
I20240502 14:00:41.371241 3539 raft_consensus.cc:2229] T da2f3b3481da476d98c97bf35f055c2e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:41.371367 3539 raft_consensus.cc:2258] T da2f3b3481da476d98c97bf35f055c2e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:41.371872 3539 ts_tablet_manager.cc:1875] T da2f3b3481da476d98c97bf35f055c2e P ad5befacd63742a1b5b69fdacdb8cf8f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:41.373468 3539 ts_tablet_manager.cc:1888] T da2f3b3481da476d98c97bf35f055c2e P ad5befacd63742a1b5b69fdacdb8cf8f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:41.373534 3539 log.cc:1196] T da2f3b3481da476d98c97bf35f055c2e P ad5befacd63742a1b5b69fdacdb8cf8f: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/wals/da2f3b3481da476d98c97bf35f055c2e
I20240502 14:00:41.374014 30698 catalog_manager.cc:4830] TS ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553): tablet da2f3b3481da476d98c97bf35f055c2e (table rebalance_test_table_6 [id=99c461286f6a4e4f81cc54ae6fc11b21]) successfully deleted
I20240502 14:00:41.444139 3561 raft_consensus.cc:1050] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929: attempting to promote NON_VOTER 3daf9e0837c74a919c89e686597ff56b to VOTER
I20240502 14:00:41.444620 3561 consensus_queue.cc:237] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:41.445945 488 raft_consensus.cc:1261] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:41.446136 31133 raft_consensus.cc:1261] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:41.446233 30865 raft_consensus.cc:1261] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:41.446571 3563 consensus_queue.cc:1035] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:41.446728 3201 consensus_queue.cc:1035] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:41.446843 3560 consensus_queue.cc:1035] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:41.449327 3201 raft_consensus.cc:2932] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:41.450946 3413 raft_consensus.cc:1050] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc: attempting to promote NON_VOTER f769c13ece0b40179ba120dbebbaba98 to VOTER
I20240502 14:00:41.451318 3413 consensus_queue.cc:237] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } }
I20240502 14:00:41.452544 31266 raft_consensus.cc:1261] T c0b9178646764824b17cbd842fbb0ff2 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:41.452441 488 raft_consensus.cc:2932] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:41.452739 30865 raft_consensus.cc:2932] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:41.452908 3349 consensus_queue.cc:1035] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:41.453007 349 raft_consensus.cc:1261] T c0b9178646764824b17cbd842fbb0ff2 P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:41.454505 31002 raft_consensus.cc:1261] T c0b9178646764824b17cbd842fbb0ff2 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:41.454768 30709 catalog_manager.cc:5484] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b reported cstate change: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:41.455134 32250 consensus_queue.cc:1035] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:41.455140 31133 raft_consensus.cc:2932] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:41.458382 32250 consensus_queue.cc:1035] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:41.461715 32250 raft_consensus.cc:2932] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:41.463306 31002 raft_consensus.cc:2932] T c0b9178646764824b17cbd842fbb0ff2 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:41.463946 30713 catalog_manager.cc:5484] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:41.464658 349 raft_consensus.cc:2932] T c0b9178646764824b17cbd842fbb0ff2 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:41.465963 31266 raft_consensus.cc:2932] T c0b9178646764824b17cbd842fbb0ff2 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:41.469803 31133 consensus_queue.cc:237] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } }
I20240502 14:00:41.470808 349 raft_consensus.cc:1261] T c0b9178646764824b17cbd842fbb0ff2 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:41.471078 31002 raft_consensus.cc:1261] T c0b9178646764824b17cbd842fbb0ff2 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:41.471662 3349 consensus_queue.cc:1035] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:41.471909 3349 consensus_queue.cc:1035] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:41.473357 3413 raft_consensus.cc:2932] T c0b9178646764824b17cbd842fbb0ff2 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:41.474480 30701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet c0b9178646764824b17cbd842fbb0ff2 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:41.475062 31002 raft_consensus.cc:2932] T c0b9178646764824b17cbd842fbb0ff2 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:41.475162 349 raft_consensus.cc:2932] T c0b9178646764824b17cbd842fbb0ff2 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:41.477228 30709 catalog_manager.cc:5484] T c0b9178646764824b17cbd842fbb0ff2 P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:41.479303 31246 tablet_service.cc:1514] Processing DeleteTablet for tablet c0b9178646764824b17cbd842fbb0ff2 with delete_type TABLET_DATA_TOMBSTONED (TS 14753c93125941acaad7283edc669052 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:53826
I20240502 14:00:41.479488 3424 tablet_replica.cc:330] T c0b9178646764824b17cbd842fbb0ff2 P 14753c93125941acaad7283edc669052: stopping tablet replica
I20240502 14:00:41.479647 3424 raft_consensus.cc:2229] T c0b9178646764824b17cbd842fbb0ff2 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:41.479815 3424 raft_consensus.cc:2258] T c0b9178646764824b17cbd842fbb0ff2 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:41.480475 3424 ts_tablet_manager.cc:1875] T c0b9178646764824b17cbd842fbb0ff2 P 14753c93125941acaad7283edc669052: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:41.481940 3424 ts_tablet_manager.cc:1888] T c0b9178646764824b17cbd842fbb0ff2 P 14753c93125941acaad7283edc669052: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:41.482015 3424 log.cc:1196] T c0b9178646764824b17cbd842fbb0ff2 P 14753c93125941acaad7283edc669052: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/c0b9178646764824b17cbd842fbb0ff2
I20240502 14:00:41.482535 30701 catalog_manager.cc:4830] TS 14753c93125941acaad7283edc669052 (127.24.66.68:37029): tablet c0b9178646764824b17cbd842fbb0ff2 (table rebalance_test_table_7 [id=2c4adac4a0a74af9bacbd4987a8a8d39]) successfully deleted
I20240502 14:00:41.498366 31002 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2e02dc2a98d04bddafdd1b933d89b4dd"
dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:53836
I20240502 14:00:41.498462 31002 raft_consensus.cc:595] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:41.525411 31002 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2e02dc2a98d04bddafdd1b933d89b4dd"
dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:53862
I20240502 14:00:41.525506 31002 raft_consensus.cc:595] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:41.560540 3364 raft_consensus.cc:1050] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052: attempting to promote NON_VOTER 3daf9e0837c74a919c89e686597ff56b to VOTER
I20240502 14:00:41.560954 3364 consensus_queue.cc:237] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:41.562263 31133 raft_consensus.cc:1261] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:41.562258 30865 raft_consensus.cc:1261] T f571119d6f624fe88bf1effeaf466085 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:41.562268 488 raft_consensus.cc:1261] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:41.562592 3362 consensus_queue.cc:1035] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:41.562774 1620 consensus_queue.cc:1035] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:41.563791 3364 consensus_queue.cc:1035] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:41.564584 1620 raft_consensus.cc:2932] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:41.564947 30865 raft_consensus.cc:2932] T f571119d6f624fe88bf1effeaf466085 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:41.566136 488 raft_consensus.cc:2932] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:41.566378 31133 raft_consensus.cc:2932] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:41.567755 30715 catalog_manager.cc:5484] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index 4 to 5, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:41.570598 31266 consensus_queue.cc:237] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:41.571559 488 raft_consensus.cc:1261] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:41.571559 31133 raft_consensus.cc:1261] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:41.571854 3362 consensus_queue.cc:1035] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:41.571992 3364 consensus_queue.cc:1035] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:41.573323 3364 raft_consensus.cc:2932] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:41.573864 488 raft_consensus.cc:2932] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:41.574640 30701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f571119d6f624fe88bf1effeaf466085 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:41.574851 31133 raft_consensus.cc:2932] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:41.576396 30709 catalog_manager.cc:5484] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index 5 to 6, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:41.578424 30843 tablet_service.cc:1514] Processing DeleteTablet for tablet f571119d6f624fe88bf1effeaf466085 with delete_type TABLET_DATA_TOMBSTONED (TS ad5befacd63742a1b5b69fdacdb8cf8f not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45544
I20240502 14:00:41.578645 3539 tablet_replica.cc:330] T f571119d6f624fe88bf1effeaf466085 P ad5befacd63742a1b5b69fdacdb8cf8f: stopping tablet replica
I20240502 14:00:41.578775 3539 raft_consensus.cc:2229] T f571119d6f624fe88bf1effeaf466085 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:41.578915 3539 raft_consensus.cc:2258] T f571119d6f624fe88bf1effeaf466085 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:41.579504 3539 ts_tablet_manager.cc:1875] T f571119d6f624fe88bf1effeaf466085 P ad5befacd63742a1b5b69fdacdb8cf8f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:41.581118 3539 ts_tablet_manager.cc:1888] T f571119d6f624fe88bf1effeaf466085 P ad5befacd63742a1b5b69fdacdb8cf8f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:41.581180 3539 log.cc:1196] T f571119d6f624fe88bf1effeaf466085 P ad5befacd63742a1b5b69fdacdb8cf8f: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/wals/f571119d6f624fe88bf1effeaf466085
I20240502 14:00:41.581612 30698 catalog_manager.cc:4830] TS ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553): tablet f571119d6f624fe88bf1effeaf466085 (table rebalance_test_table_3 [id=3fb9e83c8b7c4d90a375d14e792f650d]) successfully deleted
I20240502 14:00:41.676434 3644 raft_consensus.cc:1050] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f: attempting to promote NON_VOTER 2bb442cde32f426bb14791852dd81802 to VOTER
I20240502 14:00:41.676795 3644 consensus_queue.cc:237] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } }
I20240502 14:00:41.678074 31266 raft_consensus.cc:1261] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:41.678103 619 raft_consensus.cc:1261] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:41.678191 31002 raft_consensus.cc:1261] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:41.678474 3647 consensus_queue.cc:1035] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:41.678655 3646 consensus_queue.cc:1035] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:41.678828 3644 consensus_queue.cc:1035] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:41.680553 3646 raft_consensus.cc:2932] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:41.680964 31266 raft_consensus.cc:2932] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:41.681098 31002 raft_consensus.cc:2932] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:41.682636 619 raft_consensus.cc:2932] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:41.683537 30710 catalog_manager.cc:5484] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:41.686858 30865 consensus_queue.cc:237] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } }
I20240502 14:00:41.687844 619 raft_consensus.cc:1261] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:41.687897 31002 raft_consensus.cc:1261] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:41.688158 3647 consensus_queue.cc:1035] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:41.688498 3646 consensus_queue.cc:1035] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:41.689628 3647 raft_consensus.cc:2932] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:41.690184 619 raft_consensus.cc:2932] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:41.690212 31002 raft_consensus.cc:2932] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:41.691170 30698 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 4a12b8eaab7943c3bc9ee48c2a2307d0 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:41.691922 30715 catalog_manager.cc:5484] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index 5 to 6, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:41.693846 31246 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a12b8eaab7943c3bc9ee48c2a2307d0 with delete_type TABLET_DATA_TOMBSTONED (TS 14753c93125941acaad7283edc669052 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:53826
I20240502 14:00:41.694041 3424 tablet_replica.cc:330] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 14753c93125941acaad7283edc669052: stopping tablet replica
I20240502 14:00:41.694156 3424 raft_consensus.cc:2229] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:41.694285 3424 raft_consensus.cc:2258] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:41.694839 3424 ts_tablet_manager.cc:1875] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 14753c93125941acaad7283edc669052: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:41.696588 3424 ts_tablet_manager.cc:1888] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 14753c93125941acaad7283edc669052: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:41.696682 3424 log.cc:1196] T 4a12b8eaab7943c3bc9ee48c2a2307d0 P 14753c93125941acaad7283edc669052: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/4a12b8eaab7943c3bc9ee48c2a2307d0
I20240502 14:00:41.697096 30701 catalog_manager.cc:4830] TS 14753c93125941acaad7283edc669052 (127.24.66.68:37029): tablet 4a12b8eaab7943c3bc9ee48c2a2307d0 (table rebalance_test_table_8 [id=5cb834cd98904017a28fc7b595efc564]) successfully deleted
I20240502 14:00:41.749451 31002 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2e02dc2a98d04bddafdd1b933d89b4dd"
dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:53876
I20240502 14:00:41.749562 31002 raft_consensus.cc:595] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:41.773396 31002 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2e02dc2a98d04bddafdd1b933d89b4dd"
dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:53900
I20240502 14:00:41.773494 31002 raft_consensus.cc:595] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:41.786897 3646 raft_consensus.cc:1050] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f: attempting to promote NON_VOTER 2bb442cde32f426bb14791852dd81802 to VOTER
I20240502 14:00:41.787432 3646 consensus_queue.cc:237] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } }
I20240502 14:00:41.789024 31266 raft_consensus.cc:1261] T 903f5d6c1faf4267b7f7d7dc32670a52 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:41.789388 619 raft_consensus.cc:1261] T 903f5d6c1faf4267b7f7d7dc32670a52 P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:41.789561 31133 raft_consensus.cc:1261] T 903f5d6c1faf4267b7f7d7dc32670a52 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:00:41.789693 3644 consensus_queue.cc:1035] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:41.789999 3644 consensus_queue.cc:1035] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:41.791029 3644 consensus_queue.cc:1035] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:00:41.792145 3692 raft_consensus.cc:2932] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:41.792611 31133 raft_consensus.cc:2932] T 903f5d6c1faf4267b7f7d7dc32670a52 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:41.793213 619 raft_consensus.cc:2932] T 903f5d6c1faf4267b7f7d7dc32670a52 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:41.794668 31266 raft_consensus.cc:2932] T 903f5d6c1faf4267b7f7d7dc32670a52 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:41.795465 30713 catalog_manager.cc:5484] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index 4 to 5, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:41.798707 30865 consensus_queue.cc:237] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } }
I20240502 14:00:41.799845 619 raft_consensus.cc:1261] T 903f5d6c1faf4267b7f7d7dc32670a52 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:41.800205 3692 consensus_queue.cc:1035] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:41.800311 31266 raft_consensus.cc:1261] T 903f5d6c1faf4267b7f7d7dc32670a52 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:00:41.801316 3692 consensus_queue.cc:1035] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:41.801643 3644 raft_consensus.cc:2932] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:41.802022 619 raft_consensus.cc:2932] T 903f5d6c1faf4267b7f7d7dc32670a52 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:41.803066 30698 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 903f5d6c1faf4267b7f7d7dc32670a52 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:41.803719 30715 catalog_manager.cc:5484] T 903f5d6c1faf4267b7f7d7dc32670a52 P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:41.804477 31266 raft_consensus.cc:2932] T 903f5d6c1faf4267b7f7d7dc32670a52 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:41.806138 31115 tablet_service.cc:1514] Processing DeleteTablet for tablet 903f5d6c1faf4267b7f7d7dc32670a52 with delete_type TABLET_DATA_TOMBSTONED (TS 8c61042437444f5f92c76109dfb229fc not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:42546
I20240502 14:00:41.806499 3696 tablet_replica.cc:330] T 903f5d6c1faf4267b7f7d7dc32670a52 P 8c61042437444f5f92c76109dfb229fc: stopping tablet replica
I20240502 14:00:41.806672 3696 raft_consensus.cc:2229] T 903f5d6c1faf4267b7f7d7dc32670a52 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:41.806805 3696 raft_consensus.cc:2258] T 903f5d6c1faf4267b7f7d7dc32670a52 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:41.807411 3696 ts_tablet_manager.cc:1875] T 903f5d6c1faf4267b7f7d7dc32670a52 P 8c61042437444f5f92c76109dfb229fc: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:41.809022 3696 ts_tablet_manager.cc:1888] T 903f5d6c1faf4267b7f7d7dc32670a52 P 8c61042437444f5f92c76109dfb229fc: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:00:41.809084 3696 log.cc:1196] T 903f5d6c1faf4267b7f7d7dc32670a52 P 8c61042437444f5f92c76109dfb229fc: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/wals/903f5d6c1faf4267b7f7d7dc32670a52
I20240502 14:00:41.809526 30701 catalog_manager.cc:4830] TS 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193): tablet 903f5d6c1faf4267b7f7d7dc32670a52 (table rebalance_test_table_0 [id=a5535db3201345e78ac8467a4d987c08]) successfully deleted
I20240502 14:00:41.889283 3572 raft_consensus.cc:979] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929: : Instructing follower 8c61042437444f5f92c76109dfb229fc to start an election
I20240502 14:00:41.889427 3572 raft_consensus.cc:1067] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Signalling peer 8c61042437444f5f92c76109dfb229fc to start an election
I20240502 14:00:41.889835 31133 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2e02dc2a98d04bddafdd1b933d89b4dd"
dest_uuid: "8c61042437444f5f92c76109dfb229fc"
from {username='slave'} at 127.24.66.66:57411
I20240502 14:00:41.889976 31133 raft_consensus.cc:482] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:00:41.890029 31133 raft_consensus.cc:3037] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:41.890844 31133 raft_consensus.cc:504] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:41.891292 31133 leader_election.cc:288] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 2 election: Requested vote from peers 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395), ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553), 3daf9e0837c74a919c89e686597ff56b (127.24.66.71:35161)
I20240502 14:00:41.891549 31002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e02dc2a98d04bddafdd1b933d89b4dd" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
I20240502 14:00:41.891619 30865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e02dc2a98d04bddafdd1b933d89b4dd" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
I20240502 14:00:41.891683 31002 raft_consensus.cc:3032] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:00:41.891729 31002 raft_consensus.cc:729] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Running, Role: LEADER
I20240502 14:00:41.891772 30865 raft_consensus.cc:3037] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:41.891882 31002 consensus_queue.cc:260] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:41.892047 31002 raft_consensus.cc:3037] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:41.892907 30865 raft_consensus.cc:2454] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 2.
I20240502 14:00:41.892961 31002 raft_consensus.cc:2454] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 2.
I20240502 14:00:41.893277 31067 leader_election.cc:302] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1fab10dcfb8542fca2b965fed6c67929, 8c61042437444f5f92c76109dfb229fc, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:41.893442 32250 raft_consensus.cc:2781] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:00:41.893538 32250 raft_consensus.cc:686] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [term 2 LEADER]: Becoming Leader. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Running, Role: LEADER
I20240502 14:00:41.893672 32250 consensus_queue.cc:237] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:41.895682 30715 catalog_manager.cc:5484] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc reported cstate change: term changed from 1 to 2, leader changed from 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) to 8c61042437444f5f92c76109dfb229fc (127.24.66.67). New cstate: current_term: 2 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:41.896299 488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e02dc2a98d04bddafdd1b933d89b4dd" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "3daf9e0837c74a919c89e686597ff56b"
I20240502 14:00:41.896531 488 raft_consensus.cc:3037] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:41.897595 488 raft_consensus.cc:2454] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 2.
I20240502 14:00:42.287815 31002 raft_consensus.cc:1261] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [term 2 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 14:00:42.288174 3413 consensus_queue.cc:1035] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:42.492234 488 raft_consensus.cc:1261] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b [term 2 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 14:00:42.492661 3413 consensus_queue.cc:1035] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:42.495659 30865 raft_consensus.cc:1261] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f [term 2 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 14:00:42.496117 3727 consensus_queue.cc:1035] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:00:42.949961 31133 consensus_queue.cc:237] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:42.951078 30865 raft_consensus.cc:1261] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f [term 2 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240502 14:00:42.951092 488 raft_consensus.cc:1261] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b [term 2 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240502 14:00:42.951442 3729 consensus_queue.cc:1035] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:42.951628 3727 consensus_queue.cc:1035] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:42.953215 3413 raft_consensus.cc:2932] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [term 2 LEADER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:42.953665 488 raft_consensus.cc:2932] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:42.954794 30701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2e02dc2a98d04bddafdd1b933d89b4dd with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:42.955600 30865 raft_consensus.cc:2932] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f [term 2 FOLLOWER]: Committing config change with OpId 2.7: config changed from index 5 to 7, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:42.956020 30709 catalog_manager.cc:5484] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index 5 to 7, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New cstate: current_term: 2 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:42.958925 30982 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e02dc2a98d04bddafdd1b933d89b4dd with delete_type TABLET_DATA_TOMBSTONED (TS 1fab10dcfb8542fca2b965fed6c67929 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:58604
I20240502 14:00:42.959264 3748 tablet_replica.cc:330] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929: stopping tablet replica
I20240502 14:00:42.959414 3748 raft_consensus.cc:2229] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:42.959517 3748 raft_consensus.cc:2258] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:42.960044 3748 ts_tablet_manager.cc:1875] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:42.961807 3748 ts_tablet_manager.cc:1888] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.6
I20240502 14:00:42.961874 3748 log.cc:1196] T 2e02dc2a98d04bddafdd1b933d89b4dd P 1fab10dcfb8542fca2b965fed6c67929: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/wals/2e02dc2a98d04bddafdd1b933d89b4dd
I20240502 14:00:42.962345 30700 catalog_manager.cc:4830] TS 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395): tablet 2e02dc2a98d04bddafdd1b933d89b4dd (table rebalance_test_table_0 [id=a5535db3201345e78ac8467a4d987c08]) successfully deleted
I20240502 14:00:43.093624 32671 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:43.097489 31246 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 14:00:43.100039 600 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:43.110055 30982 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 14:00:43.116626 31115 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 14:00:43.117483 30843 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 14:00:43.127334 329 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:43.148576 449 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:43.148895 1426 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:43.183246 30865 consensus_queue.cc:237] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:43.184693 31134 raft_consensus.cc:1261] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:43.185031 619 raft_consensus.cc:1261] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:43.185638 3749 consensus_queue.cc:1035] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.185995 3647 consensus_queue.cc:1035] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.187472 3808 raft_consensus.cc:2932] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:43.187893 619 raft_consensus.cc:2932] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
W20240502 14:00:43.189618 30797 consensus_peers.cc:485] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f -> Peer 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73:34725): Couldn't send request to peer 06d9e72a952c44b391752da7674fa6c0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8da28cc4aef043baaa2282f84afd4abf. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:43.190912 30714 catalog_manager.cc:5484] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:43.191478 31134 raft_consensus.cc:2932] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:43.198926 31134 consensus_queue.cc:237] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:43.200284 31002 raft_consensus.cc:1261] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240502 14:00:43.200371 31069 consensus_peers.cc:485] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc -> Peer 3daf9e0837c74a919c89e686597ff56b (127.24.66.71:35161): Couldn't send request to peer 3daf9e0837c74a919c89e686597ff56b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2e07f0adca594453bad9433ed0017580. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:43.200395 32691 raft_consensus.cc:1261] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:43.200712 3727 consensus_queue.cc:1035] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.200846 3413 consensus_queue.cc:1035] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.202539 3729 raft_consensus.cc:2932] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:43.202971 31002 raft_consensus.cc:2932] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:43.203121 32691 raft_consensus.cc:2932] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:43.205161 30707 catalog_manager.cc:5484] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index 6 to 7, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:43.212535 31134 consensus_queue.cc:237] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:43.213706 488 raft_consensus.cc:1261] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b [term 2 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20240502 14:00:43.213866 30865 raft_consensus.cc:1261] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f [term 2 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
W20240502 14:00:43.213912 31067 consensus_peers.cc:485] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc -> Peer 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69:35027): Couldn't send request to peer 2716cee98b1b4c4591a40e1f466520e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2e02dc2a98d04bddafdd1b933d89b4dd. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:43.214152 3727 consensus_queue.cc:1035] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:43.214329 3729 consensus_queue.cc:1035] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:43.215890 3729 raft_consensus.cc:2932] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [term 2 LEADER]: Committing config change with OpId 2.8: config changed from index 7 to 8, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:43.216496 488 raft_consensus.cc:2932] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 7 to 8, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:43.216733 30865 raft_consensus.cc:2932] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f [term 2 FOLLOWER]: Committing config change with OpId 2.8: config changed from index 7 to 8, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:43.219105 30713 catalog_manager.cc:5484] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b reported cstate change: config changed from index 7 to 8, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New cstate: current_term: 2 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:43.226205 31266 consensus_queue.cc:237] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:43.227705 31134 raft_consensus.cc:1261] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:43.227804 349 raft_consensus.cc:1261] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:43.228041 3643 consensus_queue.cc:1035] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.228227 3834 consensus_queue.cc:1035] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.229945 3833 raft_consensus.cc:2932] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:43.230373 31134 raft_consensus.cc:2932] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
W20240502 14:00:43.232151 31199 consensus_peers.cc:485] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 -> Peer 2bb442cde32f426bb14791852dd81802 (127.24.66.72:34475): Couldn't send request to peer 2bb442cde32f426bb14791852dd81802. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: da2f3b3481da476d98c97bf35f055c2e. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:43.232966 30707 catalog_manager.cc:5484] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index 6 to 7, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:43.233419 349 raft_consensus.cc:2932] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:43.240681 30865 consensus_queue.cc:237] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:43.242028 31002 raft_consensus.cc:1261] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240502 14:00:43.242069 30797 consensus_peers.cc:485] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f -> Peer f769c13ece0b40179ba120dbebbaba98 (127.24.66.70:38291): Couldn't send request to peer f769c13ece0b40179ba120dbebbaba98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: aaca0594b68146cdbd18946f123693be. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:43.242242 32691 raft_consensus.cc:1261] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:43.242363 3647 consensus_queue.cc:1035] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.242624 3749 consensus_queue.cc:1035] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.244038 3647 raft_consensus.cc:2932] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:43.244532 31002 raft_consensus.cc:2932] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:43.244590 32691 raft_consensus.cc:2932] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:43.246896 30714 catalog_manager.cc:5484] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5 reported cstate change: config changed from index 6 to 7, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:43.254012 30865 consensus_queue.cc:237] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:43.255360 32691 raft_consensus.cc:1261] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:43.255376 31134 raft_consensus.cc:1261] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:43.255692 3749 consensus_queue.cc:1035] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.255923 3647 consensus_queue.cc:1035] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20240502 14:00:43.256127 30797 consensus_peers.cc:485] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f -> Peer 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73:34725): Couldn't send request to peer 06d9e72a952c44b391752da7674fa6c0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: efd8e3b804cf41bebf5e629b65571979. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:43.257617 3808 raft_consensus.cc:2932] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:43.258036 31134 raft_consensus.cc:2932] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:43.260108 30713 catalog_manager.cc:5484] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:43.260475 32691 raft_consensus.cc:2932] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:43.267524 30865 consensus_queue.cc:237] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:43.268790 31266 raft_consensus.cc:1261] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:43.268851 32691 raft_consensus.cc:1261] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240502 14:00:43.269073 30797 consensus_peers.cc:485] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f -> Peer 3daf9e0837c74a919c89e686597ff56b (127.24.66.71:35161): Couldn't send request to peer 3daf9e0837c74a919c89e686597ff56b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d0409950632d4eeead8a826034b5a9e4. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:43.269359 3749 consensus_queue.cc:1035] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.269529 3647 consensus_queue.cc:1035] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.271229 3749 raft_consensus.cc:2932] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:43.271859 31266 raft_consensus.cc:2932] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:43.273547 30713 catalog_manager.cc:5484] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index 6 to 7, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:43.275422 32690 raft_consensus.cc:2932] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } } }
I20240502 14:00:43.281342 30865 consensus_queue.cc:237] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
W20240502 14:00:43.282979 30794 consensus_peers.cc:485] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f -> Peer 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69:35027): Couldn't send request to peer 2716cee98b1b4c4591a40e1f466520e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 342cec2187904922890444a7e5dd181b. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:43.282999 619 raft_consensus.cc:1261] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:43.283005 31266 raft_consensus.cc:1261] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:43.283347 3749 consensus_queue.cc:1035] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.283524 3808 consensus_queue.cc:1035] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.285218 3749 raft_consensus.cc:2932] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:43.285650 619 raft_consensus.cc:2932] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:43.285861 31265 raft_consensus.cc:2932] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:43.288686 30714 catalog_manager.cc:5484] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index 6 to 7, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:43.301381 31266 consensus_queue.cc:237] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:43.302652 32691 raft_consensus.cc:1261] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:43.302675 31134 raft_consensus.cc:1261] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240502 14:00:43.302817 31199 consensus_peers.cc:485] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 -> Peer 2bb442cde32f426bb14791852dd81802 (127.24.66.72:34475): Couldn't send request to peer 2bb442cde32f426bb14791852dd81802. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 08588e44069d4044b2d09eb02d252f82. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:43.303020 3834 consensus_queue.cc:1035] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.303191 3643 consensus_queue.cc:1035] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.304790 3834 raft_consensus.cc:2932] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:43.305173 32691 raft_consensus.cc:2932] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:43.305320 31134 raft_consensus.cc:2932] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } } }
I20240502 14:00:43.307332 30713 catalog_manager.cc:5484] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index 6 to 7, NON_VOTER 2bb442cde32f426bb14791852dd81802 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:43.317751 31134 consensus_queue.cc:237] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:43.319064 619 raft_consensus.cc:1261] T 66e2378d5c0f400299a170df8a939577 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240502 14:00:43.319272 31069 consensus_peers.cc:485] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc -> Peer f769c13ece0b40179ba120dbebbaba98 (127.24.66.70:38291): Couldn't send request to peer f769c13ece0b40179ba120dbebbaba98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 66e2378d5c0f400299a170df8a939577. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:43.319409 3729 consensus_queue.cc:1035] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.319914 30865 raft_consensus.cc:1261] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:43.320371 3729 consensus_queue.cc:1035] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.321213 3727 raft_consensus.cc:2932] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:43.321628 619 raft_consensus.cc:2932] T 66e2378d5c0f400299a170df8a939577 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:43.322535 30865 raft_consensus.cc:2932] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } } }
I20240502 14:00:43.324016 30711 catalog_manager.cc:5484] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index 6 to 7, NON_VOTER f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) added. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:43.331763 31002 consensus_queue.cc:237] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:43.333228 31134 raft_consensus.cc:1261] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:43.333588 488 raft_consensus.cc:1261] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:43.333609 3572 consensus_queue.cc:1035] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.333959 3894 consensus_queue.cc:1035] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.335337 3572 raft_consensus.cc:2932] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:43.335798 31134 raft_consensus.cc:2932] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:43.337669 488 raft_consensus.cc:2932] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
W20240502 14:00:43.338248 30935 consensus_peers.cc:485] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 -> Peer 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73:34725): Couldn't send request to peer 06d9e72a952c44b391752da7674fa6c0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ec88a191b8d4188985b5e3c91a2a031. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:43.339376 30707 catalog_manager.cc:5484] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:43.345620 31134 consensus_queue.cc:237] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:43.346822 619 raft_consensus.cc:1261] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:43.347234 31266 raft_consensus.cc:1261] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:43.347564 3413 consensus_queue.cc:1035] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.347765 3727 consensus_queue.cc:1035] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.349663 3413 raft_consensus.cc:2932] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:43.350251 31266 raft_consensus.cc:2932] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
W20240502 14:00:43.350718 31069 consensus_peers.cc:485] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc -> Peer 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73:34725): Couldn't send request to peer 06d9e72a952c44b391752da7674fa6c0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc64ea7db2064251bed364b800b36b87. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:43.351310 619 raft_consensus.cc:2932] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:43.352979 30711 catalog_manager.cc:5484] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:43.360034 31266 consensus_queue.cc:237] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:43.361330 488 raft_consensus.cc:1261] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:43.361392 31134 raft_consensus.cc:1261] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240502 14:00:43.361501 31199 consensus_peers.cc:485] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 -> Peer 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69:35027): Couldn't send request to peer 2716cee98b1b4c4591a40e1f466520e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f571119d6f624fe88bf1effeaf466085. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:43.361763 3643 consensus_queue.cc:1035] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.361912 3833 consensus_queue.cc:1035] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.363545 3838 raft_consensus.cc:2932] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:43.363940 31134 raft_consensus.cc:2932] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:43.363971 488 raft_consensus.cc:2932] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:43.366310 30713 catalog_manager.cc:5484] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b reported cstate change: config changed from index 6 to 7, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:43.373128 31134 consensus_queue.cc:237] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:43.374526 30865 raft_consensus.cc:1261] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:43.374540 619 raft_consensus.cc:1261] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240502 14:00:43.374567 31069 consensus_peers.cc:485] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc -> Peer 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73:34725): Couldn't send request to peer 06d9e72a952c44b391752da7674fa6c0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 03fb6c61ce4c4ccdafdc9ca7a40db4f5. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:43.374897 3413 consensus_queue.cc:1035] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.375042 3727 consensus_queue.cc:1035] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.376677 3727 raft_consensus.cc:2932] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:43.378072 619 raft_consensus.cc:2932] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:43.378806 30711 catalog_manager.cc:5484] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:43.378842 30865 raft_consensus.cc:2932] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:43.387204 31266 consensus_queue.cc:237] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:43.388407 31134 raft_consensus.cc:1261] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:43.388468 488 raft_consensus.cc:1261] T 39239edeafb84b5caaa5dceea06fd4b0 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240502 14:00:43.388756 31199 consensus_peers.cc:485] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 -> Peer 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69:35027): Couldn't send request to peer 2716cee98b1b4c4591a40e1f466520e5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39239edeafb84b5caaa5dceea06fd4b0. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:43.388844 3833 consensus_queue.cc:1035] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.389012 3834 consensus_queue.cc:1035] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:43.390568 3643 raft_consensus.cc:2932] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:43.390967 31134 raft_consensus.cc:2932] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:43.393321 30711 catalog_manager.cc:5484] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index 6 to 7, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:43.393748 488 raft_consensus.cc:2932] T 39239edeafb84b5caaa5dceea06fd4b0 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } } }
I20240502 14:00:43.585621 4007 ts_tablet_manager.cc:902] T 2e07f0adca594453bad9433ed0017580 P 3daf9e0837c74a919c89e686597ff56b: Initiating tablet copy from peer 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:43.586113 4007 tablet_copy_client.cc:308] T 2e07f0adca594453bad9433ed0017580 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.67:34193
I20240502 14:00:43.589751 31155 tablet_copy_service.cc:140] P 8c61042437444f5f92c76109dfb229fc: Received BeginTabletCopySession request for tablet 2e07f0adca594453bad9433ed0017580 from peer 3daf9e0837c74a919c89e686597ff56b ({username='slave'} at 127.24.66.71:51335)
I20240502 14:00:43.589864 31155 tablet_copy_service.cc:161] P 8c61042437444f5f92c76109dfb229fc: Beginning new tablet copy session on tablet 2e07f0adca594453bad9433ed0017580 from peer 3daf9e0837c74a919c89e686597ff56b at {username='slave'} at 127.24.66.71:51335: session id = 3daf9e0837c74a919c89e686597ff56b-2e07f0adca594453bad9433ed0017580
I20240502 14:00:43.590605 31155 tablet_copy_source_session.cc:205] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:43.591418 4007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e07f0adca594453bad9433ed0017580. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.593209 4007 tablet_copy_client.cc:783] T 2e07f0adca594453bad9433ed0017580 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:43.593348 4007 tablet_copy_client.cc:647] T 2e07f0adca594453bad9433ed0017580 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:43.594276 4011 tablet_copy_client.cc:1002] T 2e07f0adca594453bad9433ed0017580 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:43.629818 4013 ts_tablet_manager.cc:902] T 8da28cc4aef043baaa2282f84afd4abf P 06d9e72a952c44b391752da7674fa6c0: Initiating tablet copy from peer ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
I20240502 14:00:43.630846 4013 tablet_copy_client.cc:308] T 8da28cc4aef043baaa2282f84afd4abf P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.65:39553
I20240502 14:00:43.634522 30887 tablet_copy_service.cc:140] P ad5befacd63742a1b5b69fdacdb8cf8f: Received BeginTabletCopySession request for tablet 8da28cc4aef043baaa2282f84afd4abf from peer 06d9e72a952c44b391752da7674fa6c0 ({username='slave'} at 127.24.66.73:39479)
I20240502 14:00:43.634635 30887 tablet_copy_service.cc:161] P ad5befacd63742a1b5b69fdacdb8cf8f: Beginning new tablet copy session on tablet 8da28cc4aef043baaa2282f84afd4abf from peer 06d9e72a952c44b391752da7674fa6c0 at {username='slave'} at 127.24.66.73:39479: session id = 06d9e72a952c44b391752da7674fa6c0-8da28cc4aef043baaa2282f84afd4abf
I20240502 14:00:43.635427 30887 tablet_copy_source_session.cc:205] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:43.638859 4013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8da28cc4aef043baaa2282f84afd4abf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.643568 4013 tablet_copy_client.cc:783] T 8da28cc4aef043baaa2282f84afd4abf P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:43.643769 4013 tablet_copy_client.cc:647] T 8da28cc4aef043baaa2282f84afd4abf P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:43.645032 4017 tablet_copy_client.cc:1002] T 8da28cc4aef043baaa2282f84afd4abf P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:43.679669 4024 ts_tablet_manager.cc:902] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b: Initiating tablet copy from peer ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
I20240502 14:00:43.680207 4024 tablet_copy_client.cc:308] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.65:39553
I20240502 14:00:43.680658 30887 tablet_copy_service.cc:140] P ad5befacd63742a1b5b69fdacdb8cf8f: Received BeginTabletCopySession request for tablet d0409950632d4eeead8a826034b5a9e4 from peer 3daf9e0837c74a919c89e686597ff56b ({username='slave'} at 127.24.66.71:50625)
I20240502 14:00:43.680752 30887 tablet_copy_service.cc:161] P ad5befacd63742a1b5b69fdacdb8cf8f: Beginning new tablet copy session on tablet d0409950632d4eeead8a826034b5a9e4 from peer 3daf9e0837c74a919c89e686597ff56b at {username='slave'} at 127.24.66.71:50625: session id = 3daf9e0837c74a919c89e686597ff56b-d0409950632d4eeead8a826034b5a9e4
I20240502 14:00:43.681715 30887 tablet_copy_source_session.cc:205] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:43.682497 4024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0409950632d4eeead8a826034b5a9e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.684298 4024 tablet_copy_client.cc:783] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:43.684486 4024 tablet_copy_client.cc:647] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:43.685606 4028 tablet_copy_client.cc:1002] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:43.693181 4044 ts_tablet_manager.cc:902] T 2e02dc2a98d04bddafdd1b933d89b4dd P 2716cee98b1b4c4591a40e1f466520e5: Initiating tablet copy from peer 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:43.693915 4044 tablet_copy_client.cc:308] T 2e02dc2a98d04bddafdd1b933d89b4dd P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.67:34193
I20240502 14:00:43.694380 31155 tablet_copy_service.cc:140] P 8c61042437444f5f92c76109dfb229fc: Received BeginTabletCopySession request for tablet 2e02dc2a98d04bddafdd1b933d89b4dd from peer 2716cee98b1b4c4591a40e1f466520e5 ({username='slave'} at 127.24.66.69:44787)
I20240502 14:00:43.694466 31155 tablet_copy_service.cc:161] P 8c61042437444f5f92c76109dfb229fc: Beginning new tablet copy session on tablet 2e02dc2a98d04bddafdd1b933d89b4dd from peer 2716cee98b1b4c4591a40e1f466520e5 at {username='slave'} at 127.24.66.69:44787: session id = 2716cee98b1b4c4591a40e1f466520e5-2e02dc2a98d04bddafdd1b933d89b4dd
I20240502 14:00:43.695358 31155 tablet_copy_source_session.cc:205] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:43.696249 4044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e02dc2a98d04bddafdd1b933d89b4dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.698551 4044 tablet_copy_client.cc:783] T 2e02dc2a98d04bddafdd1b933d89b4dd P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:43.698704 4044 tablet_copy_client.cc:647] T 2e02dc2a98d04bddafdd1b933d89b4dd P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:43.699637 4047 tablet_copy_client.cc:1002] T 2e02dc2a98d04bddafdd1b933d89b4dd P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:43.720510 4074 ts_tablet_manager.cc:902] T aaca0594b68146cdbd18946f123693be P f769c13ece0b40179ba120dbebbaba98: Initiating tablet copy from peer ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
I20240502 14:00:43.721165 4074 tablet_copy_client.cc:308] T aaca0594b68146cdbd18946f123693be P f769c13ece0b40179ba120dbebbaba98: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.65:39553
I20240502 14:00:43.721593 30887 tablet_copy_service.cc:140] P ad5befacd63742a1b5b69fdacdb8cf8f: Received BeginTabletCopySession request for tablet aaca0594b68146cdbd18946f123693be from peer f769c13ece0b40179ba120dbebbaba98 ({username='slave'} at 127.24.66.70:52845)
I20240502 14:00:43.721688 30887 tablet_copy_service.cc:161] P ad5befacd63742a1b5b69fdacdb8cf8f: Beginning new tablet copy session on tablet aaca0594b68146cdbd18946f123693be from peer f769c13ece0b40179ba120dbebbaba98 at {username='slave'} at 127.24.66.70:52845: session id = f769c13ece0b40179ba120dbebbaba98-aaca0594b68146cdbd18946f123693be
I20240502 14:00:43.722462 30887 tablet_copy_source_session.cc:205] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:43.723253 4074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aaca0594b68146cdbd18946f123693be. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.725291 4074 tablet_copy_client.cc:783] T aaca0594b68146cdbd18946f123693be P f769c13ece0b40179ba120dbebbaba98: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:43.725476 4074 tablet_copy_client.cc:647] T aaca0594b68146cdbd18946f123693be P f769c13ece0b40179ba120dbebbaba98: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:43.726440 4078 tablet_copy_client.cc:1002] T aaca0594b68146cdbd18946f123693be P f769c13ece0b40179ba120dbebbaba98: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:43.742487 4099 ts_tablet_manager.cc:902] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0: Initiating tablet copy from peer 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:43.743068 4099 tablet_copy_client.cc:308] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.66:36395
I20240502 14:00:43.746520 31023 tablet_copy_service.cc:140] P 1fab10dcfb8542fca2b965fed6c67929: Received BeginTabletCopySession request for tablet 8ec88a191b8d4188985b5e3c91a2a031 from peer 06d9e72a952c44b391752da7674fa6c0 ({username='slave'} at 127.24.66.73:41497)
I20240502 14:00:43.746621 31023 tablet_copy_service.cc:161] P 1fab10dcfb8542fca2b965fed6c67929: Beginning new tablet copy session on tablet 8ec88a191b8d4188985b5e3c91a2a031 from peer 06d9e72a952c44b391752da7674fa6c0 at {username='slave'} at 127.24.66.73:41497: session id = 06d9e72a952c44b391752da7674fa6c0-8ec88a191b8d4188985b5e3c91a2a031
I20240502 14:00:43.747447 31023 tablet_copy_source_session.cc:205] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:43.748261 4099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ec88a191b8d4188985b5e3c91a2a031. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.750207 4099 tablet_copy_client.cc:783] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:43.750380 4099 tablet_copy_client.cc:647] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:43.751892 4107 tablet_copy_client.cc:1002] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:43.762241 4114 ts_tablet_manager.cc:902] T 66e2378d5c0f400299a170df8a939577 P f769c13ece0b40179ba120dbebbaba98: Initiating tablet copy from peer 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:43.762848 4114 tablet_copy_client.cc:308] T 66e2378d5c0f400299a170df8a939577 P f769c13ece0b40179ba120dbebbaba98: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.67:34193
I20240502 14:00:43.763270 31155 tablet_copy_service.cc:140] P 8c61042437444f5f92c76109dfb229fc: Received BeginTabletCopySession request for tablet 66e2378d5c0f400299a170df8a939577 from peer f769c13ece0b40179ba120dbebbaba98 ({username='slave'} at 127.24.66.70:53053)
I20240502 14:00:43.763370 31155 tablet_copy_service.cc:161] P 8c61042437444f5f92c76109dfb229fc: Beginning new tablet copy session on tablet 66e2378d5c0f400299a170df8a939577 from peer f769c13ece0b40179ba120dbebbaba98 at {username='slave'} at 127.24.66.70:53053: session id = f769c13ece0b40179ba120dbebbaba98-66e2378d5c0f400299a170df8a939577
I20240502 14:00:43.763993 31155 tablet_copy_source_session.cc:205] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:43.764741 4114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66e2378d5c0f400299a170df8a939577. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.766335 4114 tablet_copy_client.cc:783] T 66e2378d5c0f400299a170df8a939577 P f769c13ece0b40179ba120dbebbaba98: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:43.766472 4114 tablet_copy_client.cc:647] T 66e2378d5c0f400299a170df8a939577 P f769c13ece0b40179ba120dbebbaba98: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:43.767354 4118 tablet_copy_client.cc:1002] T 66e2378d5c0f400299a170df8a939577 P f769c13ece0b40179ba120dbebbaba98: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:43.779568 4119 ts_tablet_manager.cc:902] T 342cec2187904922890444a7e5dd181b P 2716cee98b1b4c4591a40e1f466520e5: Initiating tablet copy from peer ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
I20240502 14:00:43.780016 4119 tablet_copy_client.cc:308] T 342cec2187904922890444a7e5dd181b P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.65:39553
I20240502 14:00:43.780463 30887 tablet_copy_service.cc:140] P ad5befacd63742a1b5b69fdacdb8cf8f: Received BeginTabletCopySession request for tablet 342cec2187904922890444a7e5dd181b from peer 2716cee98b1b4c4591a40e1f466520e5 ({username='slave'} at 127.24.66.69:55949)
I20240502 14:00:43.780556 30887 tablet_copy_service.cc:161] P ad5befacd63742a1b5b69fdacdb8cf8f: Beginning new tablet copy session on tablet 342cec2187904922890444a7e5dd181b from peer 2716cee98b1b4c4591a40e1f466520e5 at {username='slave'} at 127.24.66.69:55949: session id = 2716cee98b1b4c4591a40e1f466520e5-342cec2187904922890444a7e5dd181b
I20240502 14:00:43.781267 30887 tablet_copy_source_session.cc:205] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:43.781787 4124 ts_tablet_manager.cc:902] T da2f3b3481da476d98c97bf35f055c2e P 2bb442cde32f426bb14791852dd81802: Initiating tablet copy from peer 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:43.782045 4119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 342cec2187904922890444a7e5dd181b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.782275 4124 tablet_copy_client.cc:308] T da2f3b3481da476d98c97bf35f055c2e P 2bb442cde32f426bb14791852dd81802: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:37029
I20240502 14:00:43.783736 4119 tablet_copy_client.cc:783] T 342cec2187904922890444a7e5dd181b P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:43.783890 4119 tablet_copy_client.cc:647] T 342cec2187904922890444a7e5dd181b P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:43.784873 4123 tablet_copy_client.cc:1002] T 342cec2187904922890444a7e5dd181b P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:43.785985 31288 tablet_copy_service.cc:140] P 14753c93125941acaad7283edc669052: Received BeginTabletCopySession request for tablet da2f3b3481da476d98c97bf35f055c2e from peer 2bb442cde32f426bb14791852dd81802 ({username='slave'} at 127.24.66.72:60935)
I20240502 14:00:43.786067 31288 tablet_copy_service.cc:161] P 14753c93125941acaad7283edc669052: Beginning new tablet copy session on tablet da2f3b3481da476d98c97bf35f055c2e from peer 2bb442cde32f426bb14791852dd81802 at {username='slave'} at 127.24.66.72:60935: session id = 2bb442cde32f426bb14791852dd81802-da2f3b3481da476d98c97bf35f055c2e
I20240502 14:00:43.786720 31288 tablet_copy_source_session.cc:205] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:43.787436 4124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da2f3b3481da476d98c97bf35f055c2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.789415 4124 tablet_copy_client.cc:783] T da2f3b3481da476d98c97bf35f055c2e P 2bb442cde32f426bb14791852dd81802: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:43.789566 4124 tablet_copy_client.cc:647] T da2f3b3481da476d98c97bf35f055c2e P 2bb442cde32f426bb14791852dd81802: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:43.790863 4130 ts_tablet_manager.cc:902] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 06d9e72a952c44b391752da7674fa6c0: Initiating tablet copy from peer 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:43.790866 4127 tablet_copy_client.cc:1002] T da2f3b3481da476d98c97bf35f055c2e P 2bb442cde32f426bb14791852dd81802: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:43.791404 4130 tablet_copy_client.cc:308] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.67:34193
I20240502 14:00:43.794752 31155 tablet_copy_service.cc:140] P 8c61042437444f5f92c76109dfb229fc: Received BeginTabletCopySession request for tablet 03fb6c61ce4c4ccdafdc9ca7a40db4f5 from peer 06d9e72a952c44b391752da7674fa6c0 ({username='slave'} at 127.24.66.73:40101)
I20240502 14:00:43.794835 31155 tablet_copy_service.cc:161] P 8c61042437444f5f92c76109dfb229fc: Beginning new tablet copy session on tablet 03fb6c61ce4c4ccdafdc9ca7a40db4f5 from peer 06d9e72a952c44b391752da7674fa6c0 at {username='slave'} at 127.24.66.73:40101: session id = 06d9e72a952c44b391752da7674fa6c0-03fb6c61ce4c4ccdafdc9ca7a40db4f5
I20240502 14:00:43.795610 31155 tablet_copy_source_session.cc:205] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:43.796304 4130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03fb6c61ce4c4ccdafdc9ca7a40db4f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.800741 4130 tablet_copy_client.cc:783] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:43.800908 4130 tablet_copy_client.cc:647] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:43.800997 4135 ts_tablet_manager.cc:902] T cc64ea7db2064251bed364b800b36b87 P 06d9e72a952c44b391752da7674fa6c0: Initiating tablet copy from peer 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193)
I20240502 14:00:43.801518 4135 tablet_copy_client.cc:308] T cc64ea7db2064251bed364b800b36b87 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.67:34193
I20240502 14:00:43.801913 4134 tablet_copy_client.cc:1002] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:43.801934 31155 tablet_copy_service.cc:140] P 8c61042437444f5f92c76109dfb229fc: Received BeginTabletCopySession request for tablet cc64ea7db2064251bed364b800b36b87 from peer 06d9e72a952c44b391752da7674fa6c0 ({username='slave'} at 127.24.66.73:40101)
I20240502 14:00:43.802053 31155 tablet_copy_service.cc:161] P 8c61042437444f5f92c76109dfb229fc: Beginning new tablet copy session on tablet cc64ea7db2064251bed364b800b36b87 from peer 06d9e72a952c44b391752da7674fa6c0 at {username='slave'} at 127.24.66.73:40101: session id = 06d9e72a952c44b391752da7674fa6c0-cc64ea7db2064251bed364b800b36b87
I20240502 14:00:43.802798 31155 tablet_copy_source_session.cc:205] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:43.803469 4135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc64ea7db2064251bed364b800b36b87. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.805073 4135 tablet_copy_client.cc:783] T cc64ea7db2064251bed364b800b36b87 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:43.805205 4135 tablet_copy_client.cc:647] T cc64ea7db2064251bed364b800b36b87 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:43.806075 4137 tablet_copy_client.cc:1002] T cc64ea7db2064251bed364b800b36b87 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:43.827414 4140 ts_tablet_manager.cc:902] T 39239edeafb84b5caaa5dceea06fd4b0 P 2716cee98b1b4c4591a40e1f466520e5: Initiating tablet copy from peer 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:43.827914 4140 tablet_copy_client.cc:308] T 39239edeafb84b5caaa5dceea06fd4b0 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:37029
I20240502 14:00:43.828372 31288 tablet_copy_service.cc:140] P 14753c93125941acaad7283edc669052: Received BeginTabletCopySession request for tablet 39239edeafb84b5caaa5dceea06fd4b0 from peer 2716cee98b1b4c4591a40e1f466520e5 ({username='slave'} at 127.24.66.69:42815)
I20240502 14:00:43.828471 31288 tablet_copy_service.cc:161] P 14753c93125941acaad7283edc669052: Beginning new tablet copy session on tablet 39239edeafb84b5caaa5dceea06fd4b0 from peer 2716cee98b1b4c4591a40e1f466520e5 at {username='slave'} at 127.24.66.69:42815: session id = 2716cee98b1b4c4591a40e1f466520e5-39239edeafb84b5caaa5dceea06fd4b0
I20240502 14:00:43.829164 31288 tablet_copy_source_session.cc:205] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:43.829888 4140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39239edeafb84b5caaa5dceea06fd4b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.831657 4140 tablet_copy_client.cc:783] T 39239edeafb84b5caaa5dceea06fd4b0 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:43.831836 4140 tablet_copy_client.cc:647] T 39239edeafb84b5caaa5dceea06fd4b0 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:43.832495 4145 ts_tablet_manager.cc:902] T efd8e3b804cf41bebf5e629b65571979 P 06d9e72a952c44b391752da7674fa6c0: Initiating tablet copy from peer ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
I20240502 14:00:43.833058 4145 tablet_copy_client.cc:308] T efd8e3b804cf41bebf5e629b65571979 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.65:39553
I20240502 14:00:43.833348 4144 tablet_copy_client.cc:1002] T 39239edeafb84b5caaa5dceea06fd4b0 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:43.833688 30887 tablet_copy_service.cc:140] P ad5befacd63742a1b5b69fdacdb8cf8f: Received BeginTabletCopySession request for tablet efd8e3b804cf41bebf5e629b65571979 from peer 06d9e72a952c44b391752da7674fa6c0 ({username='slave'} at 127.24.66.73:39479)
I20240502 14:00:43.833793 30887 tablet_copy_service.cc:161] P ad5befacd63742a1b5b69fdacdb8cf8f: Beginning new tablet copy session on tablet efd8e3b804cf41bebf5e629b65571979 from peer 06d9e72a952c44b391752da7674fa6c0 at {username='slave'} at 127.24.66.73:39479: session id = 06d9e72a952c44b391752da7674fa6c0-efd8e3b804cf41bebf5e629b65571979
I20240502 14:00:43.834753 30887 tablet_copy_source_session.cc:205] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:43.835256 4150 ts_tablet_manager.cc:902] T 08588e44069d4044b2d09eb02d252f82 P 2bb442cde32f426bb14791852dd81802: Initiating tablet copy from peer 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:43.835541 4145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efd8e3b804cf41bebf5e629b65571979. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.835785 4150 tablet_copy_client.cc:308] T 08588e44069d4044b2d09eb02d252f82 P 2bb442cde32f426bb14791852dd81802: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:37029
I20240502 14:00:43.836266 31288 tablet_copy_service.cc:140] P 14753c93125941acaad7283edc669052: Received BeginTabletCopySession request for tablet 08588e44069d4044b2d09eb02d252f82 from peer 2bb442cde32f426bb14791852dd81802 ({username='slave'} at 127.24.66.72:60935)
I20240502 14:00:43.836371 31288 tablet_copy_service.cc:161] P 14753c93125941acaad7283edc669052: Beginning new tablet copy session on tablet 08588e44069d4044b2d09eb02d252f82 from peer 2bb442cde32f426bb14791852dd81802 at {username='slave'} at 127.24.66.72:60935: session id = 2bb442cde32f426bb14791852dd81802-08588e44069d4044b2d09eb02d252f82
I20240502 14:00:43.837051 31288 tablet_copy_source_session.cc:205] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:43.837430 4145 tablet_copy_client.cc:783] T efd8e3b804cf41bebf5e629b65571979 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:43.837605 4145 tablet_copy_client.cc:647] T efd8e3b804cf41bebf5e629b65571979 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:43.837814 4150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08588e44069d4044b2d09eb02d252f82. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.838553 4149 tablet_copy_client.cc:1002] T efd8e3b804cf41bebf5e629b65571979 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:43.839581 4150 tablet_copy_client.cc:783] T 08588e44069d4044b2d09eb02d252f82 P 2bb442cde32f426bb14791852dd81802: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:43.839726 4150 tablet_copy_client.cc:647] T 08588e44069d4044b2d09eb02d252f82 P 2bb442cde32f426bb14791852dd81802: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:43.840618 4153 tablet_copy_client.cc:1002] T 08588e44069d4044b2d09eb02d252f82 P 2bb442cde32f426bb14791852dd81802: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:43.985081 4180 ts_tablet_manager.cc:902] T f571119d6f624fe88bf1effeaf466085 P 2716cee98b1b4c4591a40e1f466520e5: Initiating tablet copy from peer 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:43.985613 4180 tablet_copy_client.cc:308] T f571119d6f624fe88bf1effeaf466085 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:37029
I20240502 14:00:43.986083 31288 tablet_copy_service.cc:140] P 14753c93125941acaad7283edc669052: Received BeginTabletCopySession request for tablet f571119d6f624fe88bf1effeaf466085 from peer 2716cee98b1b4c4591a40e1f466520e5 ({username='slave'} at 127.24.66.69:42815)
I20240502 14:00:43.986172 31288 tablet_copy_service.cc:161] P 14753c93125941acaad7283edc669052: Beginning new tablet copy session on tablet f571119d6f624fe88bf1effeaf466085 from peer 2716cee98b1b4c4591a40e1f466520e5 at {username='slave'} at 127.24.66.69:42815: session id = 2716cee98b1b4c4591a40e1f466520e5-f571119d6f624fe88bf1effeaf466085
I20240502 14:00:43.987107 31288 tablet_copy_source_session.cc:205] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:43.987877 4180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f571119d6f624fe88bf1effeaf466085. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:43.990237 4180 tablet_copy_client.cc:783] T f571119d6f624fe88bf1effeaf466085 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:43.990401 4180 tablet_copy_client.cc:647] T f571119d6f624fe88bf1effeaf466085 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:43.991500 4184 tablet_copy_client.cc:1002] T f571119d6f624fe88bf1effeaf466085 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:45.094652 4007 tablet_copy_client.cc:523] T 2e07f0adca594453bad9433ed0017580 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:45.096223 4007 tablet_bootstrap.cc:492] T 2e07f0adca594453bad9433ed0017580 P 3daf9e0837c74a919c89e686597ff56b: Bootstrap starting.
I20240502 14:00:45.122838 4007 tablet_bootstrap.cc:492] T 2e07f0adca594453bad9433ed0017580 P 3daf9e0837c74a919c89e686597ff56b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:45.123111 4007 tablet_bootstrap.cc:492] T 2e07f0adca594453bad9433ed0017580 P 3daf9e0837c74a919c89e686597ff56b: Bootstrap complete.
I20240502 14:00:45.123276 4007 ts_tablet_manager.cc:1367] T 2e07f0adca594453bad9433ed0017580 P 3daf9e0837c74a919c89e686597ff56b: Time spent bootstrapping tablet: real 0.027s user 0.023s sys 0.000s
I20240502 14:00:45.123651 4007 raft_consensus.cc:348] T 2e07f0adca594453bad9433ed0017580 P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:45.123754 4007 raft_consensus.cc:729] T 2e07f0adca594453bad9433ed0017580 P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3daf9e0837c74a919c89e686597ff56b, State: Initialized, Role: LEARNER
I20240502 14:00:45.123863 4007 consensus_queue.cc:260] T 2e07f0adca594453bad9433ed0017580 P 3daf9e0837c74a919c89e686597ff56b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:45.124210 4007 ts_tablet_manager.cc:1398] T 2e07f0adca594453bad9433ed0017580 P 3daf9e0837c74a919c89e686597ff56b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:45.124855 31155 tablet_copy_service.cc:342] P 8c61042437444f5f92c76109dfb229fc: Request end of tablet copy session 3daf9e0837c74a919c89e686597ff56b-2e07f0adca594453bad9433ed0017580 received from {username='slave'} at 127.24.66.71:51335
I20240502 14:00:45.124938 31155 tablet_copy_service.cc:433] P 8c61042437444f5f92c76109dfb229fc: ending tablet copy session 3daf9e0837c74a919c89e686597ff56b-2e07f0adca594453bad9433ed0017580 on tablet 2e07f0adca594453bad9433ed0017580 with peer 3daf9e0837c74a919c89e686597ff56b
I20240502 14:00:45.145459 4013 tablet_copy_client.cc:523] T 8da28cc4aef043baaa2282f84afd4abf P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:45.147644 4013 tablet_bootstrap.cc:492] T 8da28cc4aef043baaa2282f84afd4abf P 06d9e72a952c44b391752da7674fa6c0: Bootstrap starting.
I20240502 14:00:45.152974 4013 log.cc:824] T 8da28cc4aef043baaa2282f84afd4abf P 06d9e72a952c44b391752da7674fa6c0: Log is configured to *not* fsync() on all Append() calls
I20240502 14:00:45.174219 4013 tablet_bootstrap.cc:492] T 8da28cc4aef043baaa2282f84afd4abf P 06d9e72a952c44b391752da7674fa6c0: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:45.174510 4013 tablet_bootstrap.cc:492] T 8da28cc4aef043baaa2282f84afd4abf P 06d9e72a952c44b391752da7674fa6c0: Bootstrap complete.
I20240502 14:00:45.174791 4013 ts_tablet_manager.cc:1367] T 8da28cc4aef043baaa2282f84afd4abf P 06d9e72a952c44b391752da7674fa6c0: Time spent bootstrapping tablet: real 0.027s user 0.027s sys 0.000s
I20240502 14:00:45.176803 4013 raft_consensus.cc:348] T 8da28cc4aef043baaa2282f84afd4abf P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:45.177009 4013 raft_consensus.cc:729] T 8da28cc4aef043baaa2282f84afd4abf P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 06d9e72a952c44b391752da7674fa6c0, State: Initialized, Role: LEARNER
I20240502 14:00:45.177318 4013 consensus_queue.cc:260] T 8da28cc4aef043baaa2282f84afd4abf P 06d9e72a952c44b391752da7674fa6c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:45.177994 4013 ts_tablet_manager.cc:1398] T 8da28cc4aef043baaa2282f84afd4abf P 06d9e72a952c44b391752da7674fa6c0: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
I20240502 14:00:45.178529 30887 tablet_copy_service.cc:342] P ad5befacd63742a1b5b69fdacdb8cf8f: Request end of tablet copy session 06d9e72a952c44b391752da7674fa6c0-8da28cc4aef043baaa2282f84afd4abf received from {username='slave'} at 127.24.66.73:39479
I20240502 14:00:45.178608 30887 tablet_copy_service.cc:433] P ad5befacd63742a1b5b69fdacdb8cf8f: ending tablet copy session 06d9e72a952c44b391752da7674fa6c0-8da28cc4aef043baaa2282f84afd4abf on tablet 8da28cc4aef043baaa2282f84afd4abf with peer 06d9e72a952c44b391752da7674fa6c0
I20240502 14:00:45.185886 4024 tablet_copy_client.cc:523] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:45.187040 4024 tablet_bootstrap.cc:492] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b: Bootstrap starting.
I20240502 14:00:45.200011 4044 tablet_copy_client.cc:523] T 2e02dc2a98d04bddafdd1b933d89b4dd P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:45.201191 4044 tablet_bootstrap.cc:492] T 2e02dc2a98d04bddafdd1b933d89b4dd P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap starting.
I20240502 14:00:45.204663 4024 tablet_bootstrap.cc:492] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:45.204959 4024 tablet_bootstrap.cc:492] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b: Bootstrap complete.
I20240502 14:00:45.205144 4024 ts_tablet_manager.cc:1367] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b: Time spent bootstrapping tablet: real 0.018s user 0.016s sys 0.002s
I20240502 14:00:45.205601 4024 raft_consensus.cc:348] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:45.205760 4024 raft_consensus.cc:729] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3daf9e0837c74a919c89e686597ff56b, State: Initialized, Role: LEARNER
I20240502 14:00:45.205858 4024 consensus_queue.cc:260] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: NON_VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: true } }
I20240502 14:00:45.206709 4024 ts_tablet_manager.cc:1398] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:45.207188 30887 tablet_copy_service.cc:342] P ad5befacd63742a1b5b69fdacdb8cf8f: Request end of tablet copy session 3daf9e0837c74a919c89e686597ff56b-d0409950632d4eeead8a826034b5a9e4 received from {username='slave'} at 127.24.66.71:50625
I20240502 14:00:45.207273 30887 tablet_copy_service.cc:433] P ad5befacd63742a1b5b69fdacdb8cf8f: ending tablet copy session 3daf9e0837c74a919c89e686597ff56b-d0409950632d4eeead8a826034b5a9e4 on tablet d0409950632d4eeead8a826034b5a9e4 with peer 3daf9e0837c74a919c89e686597ff56b
I20240502 14:00:45.211221 488 raft_consensus.cc:1203] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 14:00:45.221405 488 raft_consensus.cc:1203] T 2e07f0adca594453bad9433ed0017580 P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 14:00:45.221586 4044 tablet_bootstrap.cc:492] T 2e02dc2a98d04bddafdd1b933d89b4dd P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:45.221885 4044 tablet_bootstrap.cc:492] T 2e02dc2a98d04bddafdd1b933d89b4dd P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap complete.
I20240502 14:00:45.222059 4044 ts_tablet_manager.cc:1367] T 2e02dc2a98d04bddafdd1b933d89b4dd P 2716cee98b1b4c4591a40e1f466520e5: Time spent bootstrapping tablet: real 0.021s user 0.016s sys 0.003s
I20240502 14:00:45.222419 4044 raft_consensus.cc:348] T 2e02dc2a98d04bddafdd1b933d89b4dd P 2716cee98b1b4c4591a40e1f466520e5 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:45.222522 4044 raft_consensus.cc:729] T 2e02dc2a98d04bddafdd1b933d89b4dd P 2716cee98b1b4c4591a40e1f466520e5 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 2716cee98b1b4c4591a40e1f466520e5, State: Initialized, Role: LEARNER
I20240502 14:00:45.222620 4044 consensus_queue.cc:260] T 2e02dc2a98d04bddafdd1b933d89b4dd P 2716cee98b1b4c4591a40e1f466520e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 2.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:45.222970 4044 ts_tablet_manager.cc:1398] T 2e02dc2a98d04bddafdd1b933d89b4dd P 2716cee98b1b4c4591a40e1f466520e5: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:00:45.223428 31155 tablet_copy_service.cc:342] P 8c61042437444f5f92c76109dfb229fc: Request end of tablet copy session 2716cee98b1b4c4591a40e1f466520e5-2e02dc2a98d04bddafdd1b933d89b4dd received from {username='slave'} at 127.24.66.69:44787
I20240502 14:00:45.223505 31155 tablet_copy_service.cc:433] P 8c61042437444f5f92c76109dfb229fc: ending tablet copy session 2716cee98b1b4c4591a40e1f466520e5-2e02dc2a98d04bddafdd1b933d89b4dd on tablet 2e02dc2a98d04bddafdd1b933d89b4dd with peer 2716cee98b1b4c4591a40e1f466520e5
I20240502 14:00:45.226719 4074 tablet_copy_client.cc:523] T aaca0594b68146cdbd18946f123693be P f769c13ece0b40179ba120dbebbaba98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:45.227856 4074 tablet_bootstrap.cc:492] T aaca0594b68146cdbd18946f123693be P f769c13ece0b40179ba120dbebbaba98: Bootstrap starting.
I20240502 14:00:45.246845 4074 tablet_bootstrap.cc:492] T aaca0594b68146cdbd18946f123693be P f769c13ece0b40179ba120dbebbaba98: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:45.247098 4074 tablet_bootstrap.cc:492] T aaca0594b68146cdbd18946f123693be P f769c13ece0b40179ba120dbebbaba98: Bootstrap complete.
I20240502 14:00:45.247254 4074 ts_tablet_manager.cc:1367] T aaca0594b68146cdbd18946f123693be P f769c13ece0b40179ba120dbebbaba98: Time spent bootstrapping tablet: real 0.019s user 0.018s sys 0.000s
I20240502 14:00:45.247615 4074 raft_consensus.cc:348] T aaca0594b68146cdbd18946f123693be P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:45.247758 4074 raft_consensus.cc:729] T aaca0594b68146cdbd18946f123693be P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f769c13ece0b40179ba120dbebbaba98, State: Initialized, Role: LEARNER
I20240502 14:00:45.247855 4074 consensus_queue.cc:260] T aaca0594b68146cdbd18946f123693be P f769c13ece0b40179ba120dbebbaba98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:45.248207 4074 ts_tablet_manager.cc:1398] T aaca0594b68146cdbd18946f123693be P f769c13ece0b40179ba120dbebbaba98: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 14:00:45.248688 30887 tablet_copy_service.cc:342] P ad5befacd63742a1b5b69fdacdb8cf8f: Request end of tablet copy session f769c13ece0b40179ba120dbebbaba98-aaca0594b68146cdbd18946f123693be received from {username='slave'} at 127.24.66.70:52845
I20240502 14:00:45.248771 30887 tablet_copy_service.cc:433] P ad5befacd63742a1b5b69fdacdb8cf8f: ending tablet copy session f769c13ece0b40179ba120dbebbaba98-aaca0594b68146cdbd18946f123693be on tablet aaca0594b68146cdbd18946f123693be with peer f769c13ece0b40179ba120dbebbaba98
I20240502 14:00:45.252140 4099 tablet_copy_client.cc:523] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:45.253553 4099 tablet_bootstrap.cc:492] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0: Bootstrap starting.
I20240502 14:00:45.267678 4114 tablet_copy_client.cc:523] T 66e2378d5c0f400299a170df8a939577 P f769c13ece0b40179ba120dbebbaba98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:45.268920 4114 tablet_bootstrap.cc:492] T 66e2378d5c0f400299a170df8a939577 P f769c13ece0b40179ba120dbebbaba98: Bootstrap starting.
I20240502 14:00:45.270943 4099 tablet_bootstrap.cc:492] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:45.271203 4099 tablet_bootstrap.cc:492] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0: Bootstrap complete.
I20240502 14:00:45.271347 4099 ts_tablet_manager.cc:1367] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0: Time spent bootstrapping tablet: real 0.018s user 0.013s sys 0.003s
I20240502 14:00:45.271765 4099 raft_consensus.cc:348] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:45.271864 4099 raft_consensus.cc:729] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 06d9e72a952c44b391752da7674fa6c0, State: Initialized, Role: LEARNER
I20240502 14:00:45.271960 4099 consensus_queue.cc:260] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:45.272164 4099 ts_tablet_manager.cc:1398] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:45.272658 31023 tablet_copy_service.cc:342] P 1fab10dcfb8542fca2b965fed6c67929: Request end of tablet copy session 06d9e72a952c44b391752da7674fa6c0-8ec88a191b8d4188985b5e3c91a2a031 received from {username='slave'} at 127.24.66.73:41497
I20240502 14:00:45.272747 31023 tablet_copy_service.cc:433] P 1fab10dcfb8542fca2b965fed6c67929: ending tablet copy session 06d9e72a952c44b391752da7674fa6c0-8ec88a191b8d4188985b5e3c91a2a031 on tablet 8ec88a191b8d4188985b5e3c91a2a031 with peer 06d9e72a952c44b391752da7674fa6c0
I20240502 14:00:45.285179 4119 tablet_copy_client.cc:523] T 342cec2187904922890444a7e5dd181b P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:45.286564 4119 tablet_bootstrap.cc:492] T 342cec2187904922890444a7e5dd181b P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap starting.
I20240502 14:00:45.288112 1430 raft_consensus.cc:1203] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 14:00:45.290287 4114 tablet_bootstrap.cc:492] T 66e2378d5c0f400299a170df8a939577 P f769c13ece0b40179ba120dbebbaba98: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:45.290755 4114 tablet_bootstrap.cc:492] T 66e2378d5c0f400299a170df8a939577 P f769c13ece0b40179ba120dbebbaba98: Bootstrap complete.
I20240502 14:00:45.290911 4114 ts_tablet_manager.cc:1367] T 66e2378d5c0f400299a170df8a939577 P f769c13ece0b40179ba120dbebbaba98: Time spent bootstrapping tablet: real 0.022s user 0.016s sys 0.004s
I20240502 14:00:45.291138 4124 tablet_copy_client.cc:523] T da2f3b3481da476d98c97bf35f055c2e P 2bb442cde32f426bb14791852dd81802: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:45.291347 4114 raft_consensus.cc:348] T 66e2378d5c0f400299a170df8a939577 P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:45.291442 4114 raft_consensus.cc:729] T 66e2378d5c0f400299a170df8a939577 P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f769c13ece0b40179ba120dbebbaba98, State: Initialized, Role: LEARNER
I20240502 14:00:45.291518 4114 consensus_queue.cc:260] T 66e2378d5c0f400299a170df8a939577 P f769c13ece0b40179ba120dbebbaba98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: NON_VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: true } }
I20240502 14:00:45.292100 4114 ts_tablet_manager.cc:1398] T 66e2378d5c0f400299a170df8a939577 P f769c13ece0b40179ba120dbebbaba98: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:45.292429 4124 tablet_bootstrap.cc:492] T da2f3b3481da476d98c97bf35f055c2e P 2bb442cde32f426bb14791852dd81802: Bootstrap starting.
I20240502 14:00:45.293411 31155 tablet_copy_service.cc:342] P 8c61042437444f5f92c76109dfb229fc: Request end of tablet copy session f769c13ece0b40179ba120dbebbaba98-66e2378d5c0f400299a170df8a939577 received from {username='slave'} at 127.24.66.70:53053
I20240502 14:00:45.293480 31155 tablet_copy_service.cc:433] P 8c61042437444f5f92c76109dfb229fc: ending tablet copy session f769c13ece0b40179ba120dbebbaba98-66e2378d5c0f400299a170df8a939577 on tablet 66e2378d5c0f400299a170df8a939577 with peer f769c13ece0b40179ba120dbebbaba98
I20240502 14:00:45.302158 4130 tablet_copy_client.cc:523] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:45.303376 4130 tablet_bootstrap.cc:492] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 06d9e72a952c44b391752da7674fa6c0: Bootstrap starting.
I20240502 14:00:45.306303 4135 tablet_copy_client.cc:523] T cc64ea7db2064251bed364b800b36b87 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:45.307505 4135 tablet_bootstrap.cc:492] T cc64ea7db2064251bed364b800b36b87 P 06d9e72a952c44b391752da7674fa6c0: Bootstrap starting.
I20240502 14:00:45.308290 4119 tablet_bootstrap.cc:492] T 342cec2187904922890444a7e5dd181b P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:45.308560 4119 tablet_bootstrap.cc:492] T 342cec2187904922890444a7e5dd181b P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap complete.
I20240502 14:00:45.308735 4119 ts_tablet_manager.cc:1367] T 342cec2187904922890444a7e5dd181b P 2716cee98b1b4c4591a40e1f466520e5: Time spent bootstrapping tablet: real 0.022s user 0.019s sys 0.004s
I20240502 14:00:45.309134 4119 raft_consensus.cc:348] T 342cec2187904922890444a7e5dd181b P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:45.309270 4119 raft_consensus.cc:729] T 342cec2187904922890444a7e5dd181b P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2716cee98b1b4c4591a40e1f466520e5, State: Initialized, Role: LEARNER
I20240502 14:00:45.309368 4119 consensus_queue.cc:260] T 342cec2187904922890444a7e5dd181b P 2716cee98b1b4c4591a40e1f466520e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:45.309625 4119 ts_tablet_manager.cc:1398] T 342cec2187904922890444a7e5dd181b P 2716cee98b1b4c4591a40e1f466520e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:45.312819 30887 tablet_copy_service.cc:342] P ad5befacd63742a1b5b69fdacdb8cf8f: Request end of tablet copy session 2716cee98b1b4c4591a40e1f466520e5-342cec2187904922890444a7e5dd181b received from {username='slave'} at 127.24.66.69:55949
I20240502 14:00:45.312906 30887 tablet_copy_service.cc:433] P ad5befacd63742a1b5b69fdacdb8cf8f: ending tablet copy session 2716cee98b1b4c4591a40e1f466520e5-342cec2187904922890444a7e5dd181b on tablet 342cec2187904922890444a7e5dd181b with peer 2716cee98b1b4c4591a40e1f466520e5
I20240502 14:00:45.314769 4124 tablet_bootstrap.cc:492] T da2f3b3481da476d98c97bf35f055c2e P 2bb442cde32f426bb14791852dd81802: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:45.315078 4124 tablet_bootstrap.cc:492] T da2f3b3481da476d98c97bf35f055c2e P 2bb442cde32f426bb14791852dd81802: Bootstrap complete.
I20240502 14:00:45.315260 4124 ts_tablet_manager.cc:1367] T da2f3b3481da476d98c97bf35f055c2e P 2bb442cde32f426bb14791852dd81802: Time spent bootstrapping tablet: real 0.023s user 0.018s sys 0.004s
I20240502 14:00:45.315696 4124 raft_consensus.cc:348] T da2f3b3481da476d98c97bf35f055c2e P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:45.315847 4124 raft_consensus.cc:729] T da2f3b3481da476d98c97bf35f055c2e P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2bb442cde32f426bb14791852dd81802, State: Initialized, Role: LEARNER
I20240502 14:00:45.315974 4124 consensus_queue.cc:260] T da2f3b3481da476d98c97bf35f055c2e P 2bb442cde32f426bb14791852dd81802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:45.316309 4124 ts_tablet_manager.cc:1398] T da2f3b3481da476d98c97bf35f055c2e P 2bb442cde32f426bb14791852dd81802: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:45.318024 31288 tablet_copy_service.cc:342] P 14753c93125941acaad7283edc669052: Request end of tablet copy session 2bb442cde32f426bb14791852dd81802-da2f3b3481da476d98c97bf35f055c2e received from {username='slave'} at 127.24.66.72:60935
I20240502 14:00:45.318107 31288 tablet_copy_service.cc:433] P 14753c93125941acaad7283edc669052: ending tablet copy session 2bb442cde32f426bb14791852dd81802-da2f3b3481da476d98c97bf35f055c2e on tablet da2f3b3481da476d98c97bf35f055c2e with peer 2bb442cde32f426bb14791852dd81802
I20240502 14:00:45.322325 32690 raft_consensus.cc:1203] T 342cec2187904922890444a7e5dd181b P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 14:00:45.327575 4130 tablet_bootstrap.cc:492] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 06d9e72a952c44b391752da7674fa6c0: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:45.327808 4130 tablet_bootstrap.cc:492] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 06d9e72a952c44b391752da7674fa6c0: Bootstrap complete.
I20240502 14:00:45.327975 4130 ts_tablet_manager.cc:1367] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 06d9e72a952c44b391752da7674fa6c0: Time spent bootstrapping tablet: real 0.025s user 0.018s sys 0.004s
I20240502 14:00:45.328513 4130 raft_consensus.cc:348] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:45.328611 4130 raft_consensus.cc:729] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 06d9e72a952c44b391752da7674fa6c0, State: Initialized, Role: LEARNER
I20240502 14:00:45.328692 4130 consensus_queue.cc:260] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 06d9e72a952c44b391752da7674fa6c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:45.328945 4130 ts_tablet_manager.cc:1398] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 06d9e72a952c44b391752da7674fa6c0: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 14:00:45.329466 31155 tablet_copy_service.cc:342] P 8c61042437444f5f92c76109dfb229fc: Request end of tablet copy session 06d9e72a952c44b391752da7674fa6c0-03fb6c61ce4c4ccdafdc9ca7a40db4f5 received from {username='slave'} at 127.24.66.73:40101
I20240502 14:00:45.329555 31155 tablet_copy_service.cc:433] P 8c61042437444f5f92c76109dfb229fc: ending tablet copy session 06d9e72a952c44b391752da7674fa6c0-03fb6c61ce4c4ccdafdc9ca7a40db4f5 on tablet 03fb6c61ce4c4ccdafdc9ca7a40db4f5 with peer 06d9e72a952c44b391752da7674fa6c0
I20240502 14:00:45.333652 4140 tablet_copy_client.cc:523] T 39239edeafb84b5caaa5dceea06fd4b0 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:45.334328 4135 tablet_bootstrap.cc:492] T cc64ea7db2064251bed364b800b36b87 P 06d9e72a952c44b391752da7674fa6c0: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:45.334627 4135 tablet_bootstrap.cc:492] T cc64ea7db2064251bed364b800b36b87 P 06d9e72a952c44b391752da7674fa6c0: Bootstrap complete.
I20240502 14:00:45.334815 4135 ts_tablet_manager.cc:1367] T cc64ea7db2064251bed364b800b36b87 P 06d9e72a952c44b391752da7674fa6c0: Time spent bootstrapping tablet: real 0.027s user 0.023s sys 0.001s
I20240502 14:00:45.334980 4140 tablet_bootstrap.cc:492] T 39239edeafb84b5caaa5dceea06fd4b0 P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap starting.
I20240502 14:00:45.335194 4135 raft_consensus.cc:348] T cc64ea7db2064251bed364b800b36b87 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:45.335343 4135 raft_consensus.cc:729] T cc64ea7db2064251bed364b800b36b87 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 06d9e72a952c44b391752da7674fa6c0, State: Initialized, Role: LEARNER
I20240502 14:00:45.335450 4135 consensus_queue.cc:260] T cc64ea7db2064251bed364b800b36b87 P 06d9e72a952c44b391752da7674fa6c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:45.335700 4135 ts_tablet_manager.cc:1398] T cc64ea7db2064251bed364b800b36b87 P 06d9e72a952c44b391752da7674fa6c0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:45.336221 31155 tablet_copy_service.cc:342] P 8c61042437444f5f92c76109dfb229fc: Request end of tablet copy session 06d9e72a952c44b391752da7674fa6c0-cc64ea7db2064251bed364b800b36b87 received from {username='slave'} at 127.24.66.73:40101
I20240502 14:00:45.336313 31155 tablet_copy_service.cc:433] P 8c61042437444f5f92c76109dfb229fc: ending tablet copy session 06d9e72a952c44b391752da7674fa6c0-cc64ea7db2064251bed364b800b36b87 on tablet cc64ea7db2064251bed364b800b36b87 with peer 06d9e72a952c44b391752da7674fa6c0
I20240502 14:00:45.338798 4145 tablet_copy_client.cc:523] T efd8e3b804cf41bebf5e629b65571979 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:45.339534 349 raft_consensus.cc:1203] T aaca0594b68146cdbd18946f123693be P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 14:00:45.339977 4145 tablet_bootstrap.cc:492] T efd8e3b804cf41bebf5e629b65571979 P 06d9e72a952c44b391752da7674fa6c0: Bootstrap starting.
I20240502 14:00:45.341406 4150 tablet_copy_client.cc:523] T 08588e44069d4044b2d09eb02d252f82 P 2bb442cde32f426bb14791852dd81802: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:45.342592 4150 tablet_bootstrap.cc:492] T 08588e44069d4044b2d09eb02d252f82 P 2bb442cde32f426bb14791852dd81802: Bootstrap starting.
I20240502 14:00:45.357582 4145 tablet_bootstrap.cc:492] T efd8e3b804cf41bebf5e629b65571979 P 06d9e72a952c44b391752da7674fa6c0: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:45.357820 4145 tablet_bootstrap.cc:492] T efd8e3b804cf41bebf5e629b65571979 P 06d9e72a952c44b391752da7674fa6c0: Bootstrap complete.
I20240502 14:00:45.357968 4145 ts_tablet_manager.cc:1367] T efd8e3b804cf41bebf5e629b65571979 P 06d9e72a952c44b391752da7674fa6c0: Time spent bootstrapping tablet: real 0.018s user 0.015s sys 0.002s
I20240502 14:00:45.358291 4145 raft_consensus.cc:348] T efd8e3b804cf41bebf5e629b65571979 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:45.358389 4145 raft_consensus.cc:729] T efd8e3b804cf41bebf5e629b65571979 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 06d9e72a952c44b391752da7674fa6c0, State: Initialized, Role: LEARNER
I20240502 14:00:45.358508 4145 consensus_queue.cc:260] T efd8e3b804cf41bebf5e629b65571979 P 06d9e72a952c44b391752da7674fa6c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:45.358825 4145 ts_tablet_manager.cc:1398] T efd8e3b804cf41bebf5e629b65571979 P 06d9e72a952c44b391752da7674fa6c0: Time spent starting tablet: real 0.001s user 0.002s sys 0.001s
I20240502 14:00:45.359272 30887 tablet_copy_service.cc:342] P ad5befacd63742a1b5b69fdacdb8cf8f: Request end of tablet copy session 06d9e72a952c44b391752da7674fa6c0-efd8e3b804cf41bebf5e629b65571979 received from {username='slave'} at 127.24.66.73:39479
I20240502 14:00:45.359354 30887 tablet_copy_service.cc:433] P ad5befacd63742a1b5b69fdacdb8cf8f: ending tablet copy session 06d9e72a952c44b391752da7674fa6c0-efd8e3b804cf41bebf5e629b65571979 on tablet efd8e3b804cf41bebf5e629b65571979 with peer 06d9e72a952c44b391752da7674fa6c0
I20240502 14:00:45.359671 4140 tablet_bootstrap.cc:492] T 39239edeafb84b5caaa5dceea06fd4b0 P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:45.359949 4140 tablet_bootstrap.cc:492] T 39239edeafb84b5caaa5dceea06fd4b0 P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap complete.
I20240502 14:00:45.360118 4140 ts_tablet_manager.cc:1367] T 39239edeafb84b5caaa5dceea06fd4b0 P 2716cee98b1b4c4591a40e1f466520e5: Time spent bootstrapping tablet: real 0.025s user 0.021s sys 0.005s
I20240502 14:00:45.360569 4140 raft_consensus.cc:348] T 39239edeafb84b5caaa5dceea06fd4b0 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:45.360715 4140 raft_consensus.cc:729] T 39239edeafb84b5caaa5dceea06fd4b0 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2716cee98b1b4c4591a40e1f466520e5, State: Initialized, Role: LEARNER
I20240502 14:00:45.360836 4140 consensus_queue.cc:260] T 39239edeafb84b5caaa5dceea06fd4b0 P 2716cee98b1b4c4591a40e1f466520e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:45.361137 4140 ts_tablet_manager.cc:1398] T 39239edeafb84b5caaa5dceea06fd4b0 P 2716cee98b1b4c4591a40e1f466520e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:45.361632 31288 tablet_copy_service.cc:342] P 14753c93125941acaad7283edc669052: Request end of tablet copy session 2716cee98b1b4c4591a40e1f466520e5-39239edeafb84b5caaa5dceea06fd4b0 received from {username='slave'} at 127.24.66.69:42815
I20240502 14:00:45.361702 31288 tablet_copy_service.cc:433] P 14753c93125941acaad7283edc669052: ending tablet copy session 2716cee98b1b4c4591a40e1f466520e5-39239edeafb84b5caaa5dceea06fd4b0 on tablet 39239edeafb84b5caaa5dceea06fd4b0 with peer 2716cee98b1b4c4591a40e1f466520e5
I20240502 14:00:45.369484 4150 tablet_bootstrap.cc:492] T 08588e44069d4044b2d09eb02d252f82 P 2bb442cde32f426bb14791852dd81802: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:45.369721 4150 tablet_bootstrap.cc:492] T 08588e44069d4044b2d09eb02d252f82 P 2bb442cde32f426bb14791852dd81802: Bootstrap complete.
I20240502 14:00:45.369908 4150 ts_tablet_manager.cc:1367] T 08588e44069d4044b2d09eb02d252f82 P 2bb442cde32f426bb14791852dd81802: Time spent bootstrapping tablet: real 0.027s user 0.026s sys 0.000s
I20240502 14:00:45.370270 4150 raft_consensus.cc:348] T 08588e44069d4044b2d09eb02d252f82 P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:45.370359 4150 raft_consensus.cc:729] T 08588e44069d4044b2d09eb02d252f82 P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2bb442cde32f426bb14791852dd81802, State: Initialized, Role: LEARNER
I20240502 14:00:45.370467 4150 consensus_queue.cc:260] T 08588e44069d4044b2d09eb02d252f82 P 2bb442cde32f426bb14791852dd81802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: true } }
I20240502 14:00:45.370745 4150 ts_tablet_manager.cc:1398] T 08588e44069d4044b2d09eb02d252f82 P 2bb442cde32f426bb14791852dd81802: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:45.371135 31288 tablet_copy_service.cc:342] P 14753c93125941acaad7283edc669052: Request end of tablet copy session 2bb442cde32f426bb14791852dd81802-08588e44069d4044b2d09eb02d252f82 received from {username='slave'} at 127.24.66.72:60935
I20240502 14:00:45.371204 31288 tablet_copy_service.cc:433] P 14753c93125941acaad7283edc669052: ending tablet copy session 2bb442cde32f426bb14791852dd81802-08588e44069d4044b2d09eb02d252f82 on tablet 08588e44069d4044b2d09eb02d252f82 with peer 2bb442cde32f426bb14791852dd81802
I20240502 14:00:45.453552 32690 raft_consensus.cc:1203] T 39239edeafb84b5caaa5dceea06fd4b0 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 14:00:45.463888 619 raft_consensus.cc:1203] T da2f3b3481da476d98c97bf35f055c2e P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 14:00:45.472425 1430 raft_consensus.cc:1203] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 14:00:45.491745 4180 tablet_copy_client.cc:523] T f571119d6f624fe88bf1effeaf466085 P 2716cee98b1b4c4591a40e1f466520e5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:45.492897 4180 tablet_bootstrap.cc:492] T f571119d6f624fe88bf1effeaf466085 P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap starting.
I20240502 14:00:45.509727 4180 tablet_bootstrap.cc:492] T f571119d6f624fe88bf1effeaf466085 P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:45.509958 4180 tablet_bootstrap.cc:492] T f571119d6f624fe88bf1effeaf466085 P 2716cee98b1b4c4591a40e1f466520e5: Bootstrap complete.
I20240502 14:00:45.510108 4180 ts_tablet_manager.cc:1367] T f571119d6f624fe88bf1effeaf466085 P 2716cee98b1b4c4591a40e1f466520e5: Time spent bootstrapping tablet: real 0.017s user 0.014s sys 0.002s
I20240502 14:00:45.510519 4180 raft_consensus.cc:348] T f571119d6f624fe88bf1effeaf466085 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:45.510656 4180 raft_consensus.cc:729] T f571119d6f624fe88bf1effeaf466085 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2716cee98b1b4c4591a40e1f466520e5, State: Initialized, Role: LEARNER
I20240502 14:00:45.510787 4180 consensus_queue.cc:260] T f571119d6f624fe88bf1effeaf466085 P 2716cee98b1b4c4591a40e1f466520e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: true } }
I20240502 14:00:45.510915 1430 raft_consensus.cc:1203] T 8da28cc4aef043baaa2282f84afd4abf P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 14:00:45.511040 4180 ts_tablet_manager.cc:1398] T f571119d6f624fe88bf1effeaf466085 P 2716cee98b1b4c4591a40e1f466520e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:00:45.511700 31288 tablet_copy_service.cc:342] P 14753c93125941acaad7283edc669052: Request end of tablet copy session 2716cee98b1b4c4591a40e1f466520e5-f571119d6f624fe88bf1effeaf466085 received from {username='slave'} at 127.24.66.69:42815
I20240502 14:00:45.511781 31288 tablet_copy_service.cc:433] P 14753c93125941acaad7283edc669052: ending tablet copy session 2716cee98b1b4c4591a40e1f466520e5-f571119d6f624fe88bf1effeaf466085 on tablet f571119d6f624fe88bf1effeaf466085 with peer 2716cee98b1b4c4591a40e1f466520e5
I20240502 14:00:45.547186 349 raft_consensus.cc:1203] T 66e2378d5c0f400299a170df8a939577 P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 14:00:45.628715 619 raft_consensus.cc:1203] T 08588e44069d4044b2d09eb02d252f82 P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 14:00:45.673030 4674 raft_consensus.cc:1050] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929: attempting to promote NON_VOTER 06d9e72a952c44b391752da7674fa6c0 to VOTER
I20240502 14:00:45.673427 4674 consensus_queue.cc:237] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:45.674557 31134 raft_consensus.cc:1261] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:45.674711 1430 raft_consensus.cc:1261] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:45.675047 488 raft_consensus.cc:1261] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:45.674947 4682 consensus_queue.cc:1035] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:45.675977 4392 consensus_queue.cc:1035] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:45.676584 4681 consensus_queue.cc:1035] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:45.678125 4682 raft_consensus.cc:2932] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:45.678512 31134 raft_consensus.cc:2932] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:45.680506 1430 raft_consensus.cc:2932] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:45.681150 30707 catalog_manager.cc:5484] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:45.683174 488 raft_consensus.cc:2932] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:45.718868 32690 raft_consensus.cc:1203] T 2e02dc2a98d04bddafdd1b933d89b4dd P 2716cee98b1b4c4591a40e1f466520e5 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.7->[2.8-2.8] Dedup: 2.8->[]
I20240502 14:00:45.759968 1430 raft_consensus.cc:1203] T efd8e3b804cf41bebf5e629b65571979 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 14:00:45.761400 4235 raft_consensus.cc:1050] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f: attempting to promote NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 to VOTER
I20240502 14:00:45.761852 4235 consensus_queue.cc:237] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } }
I20240502 14:00:45.763168 32690 raft_consensus.cc:1261] T 342cec2187904922890444a7e5dd181b P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:45.763242 31266 raft_consensus.cc:1261] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:45.763339 619 raft_consensus.cc:1261] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:45.763604 4729 consensus_queue.cc:1035] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:45.765220 4727 consensus_queue.cc:1035] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:45.765486 4726 consensus_queue.cc:1035] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:45.767448 4727 raft_consensus.cc:2932] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:45.767827 619 raft_consensus.cc:2932] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:45.768074 31266 raft_consensus.cc:2932] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:45.770215 32690 raft_consensus.cc:2932] T 342cec2187904922890444a7e5dd181b P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:45.770356 30714 catalog_manager.cc:5484] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index 7 to 8, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:45.773679 30865 consensus_queue.cc:237] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } }
I20240502 14:00:45.774600 32690 raft_consensus.cc:1261] T 342cec2187904922890444a7e5dd181b P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:45.774921 619 raft_consensus.cc:1261] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:45.774981 4235 consensus_queue.cc:1035] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:45.775241 4729 consensus_queue.cc:1035] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:45.775635 4729 raft_consensus.cc:1050] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f: attempting to promote NON_VOTER f769c13ece0b40179ba120dbebbaba98 to VOTER
I20240502 14:00:45.776018 4729 consensus_queue.cc:237] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } }
I20240502 14:00:45.777449 349 raft_consensus.cc:1261] T aaca0594b68146cdbd18946f123693be P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:45.777503 4235 raft_consensus.cc:2932] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:45.777781 4727 consensus_queue.cc:1035] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:45.777856 619 raft_consensus.cc:2932] T 342cec2187904922890444a7e5dd181b P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:45.777976 32691 raft_consensus.cc:2932] T 342cec2187904922890444a7e5dd181b P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:45.779006 31001 raft_consensus.cc:1261] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:45.779422 4729 consensus_queue.cc:1035] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:45.779826 30713 catalog_manager.cc:5484] T 342cec2187904922890444a7e5dd181b P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index 8 to 9, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:45.780023 30698 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 342cec2187904922890444a7e5dd181b with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:45.780251 32690 raft_consensus.cc:1261] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:45.780570 4726 consensus_queue.cc:1035] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:45.782181 4729 raft_consensus.cc:2932] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:45.782472 31246 tablet_service.cc:1514] Processing DeleteTablet for tablet 342cec2187904922890444a7e5dd181b with delete_type TABLET_DATA_TOMBSTONED (TS 14753c93125941acaad7283edc669052 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:53826
I20240502 14:00:45.782761 4737 tablet_replica.cc:330] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052: stopping tablet replica
I20240502 14:00:45.782878 4737 raft_consensus.cc:2229] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:45.783061 4737 raft_consensus.cc:2258] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:45.783699 4737 ts_tablet_manager.cc:1875] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:45.783908 349 raft_consensus.cc:2932] T aaca0594b68146cdbd18946f123693be P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:45.786432 32690 raft_consensus.cc:2932] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:45.786659 4737 ts_tablet_manager.cc:1888] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 14:00:45.786770 4737 log.cc:1196] T 342cec2187904922890444a7e5dd181b P 14753c93125941acaad7283edc669052: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/342cec2187904922890444a7e5dd181b
I20240502 14:00:45.787312 30701 catalog_manager.cc:4830] TS 14753c93125941acaad7283edc669052 (127.24.66.68:37029): tablet 342cec2187904922890444a7e5dd181b (table rebalance_test_table_9 [id=5f28187b5b7a472f8429ed627f138f76]) successfully deleted
I20240502 14:00:45.788687 30707 catalog_manager.cc:5484] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5 reported cstate change: config changed from index 7 to 8, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:45.790416 31001 raft_consensus.cc:2932] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:45.795364 30865 consensus_queue.cc:237] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } }
I20240502 14:00:45.796247 349 raft_consensus.cc:1261] T aaca0594b68146cdbd18946f123693be P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:45.796383 32690 raft_consensus.cc:1261] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:45.796649 4739 consensus_queue.cc:1035] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:45.796800 4738 consensus_queue.cc:1035] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:45.798116 4727 raft_consensus.cc:2932] T aaca0594b68146cdbd18946f123693be P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:45.798547 349 raft_consensus.cc:2932] T aaca0594b68146cdbd18946f123693be P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:45.798553 32690 raft_consensus.cc:2932] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:45.802085 30698 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet aaca0594b68146cdbd18946f123693be with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:45.801992 30713 catalog_manager.cc:5484] T aaca0594b68146cdbd18946f123693be P 2716cee98b1b4c4591a40e1f466520e5 reported cstate change: config changed from index 8 to 9, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:45.802649 4255 raft_consensus.cc:1050] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc: attempting to promote NON_VOTER 3daf9e0837c74a919c89e686597ff56b to VOTER
I20240502 14:00:45.803723 4255 consensus_queue.cc:237] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:45.805629 32690 raft_consensus.cc:1261] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:45.805771 488 raft_consensus.cc:1261] T 2e07f0adca594453bad9433ed0017580 P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:45.806124 4255 consensus_queue.cc:1035] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:45.806186 31002 raft_consensus.cc:1261] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:45.806594 4741 consensus_queue.cc:1035] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:45.807013 4743 consensus_queue.cc:1035] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:45.807168 30982 tablet_service.cc:1514] Processing DeleteTablet for tablet aaca0594b68146cdbd18946f123693be with delete_type TABLET_DATA_TOMBSTONED (TS 1fab10dcfb8542fca2b965fed6c67929 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:58604
I20240502 14:00:45.808606 4255 raft_consensus.cc:2932] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:45.808935 4745 tablet_replica.cc:330] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929: stopping tablet replica
I20240502 14:00:45.808974 31002 raft_consensus.cc:2932] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:45.809090 4745 raft_consensus.cc:2229] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:45.809307 4745 raft_consensus.cc:2258] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:45.809988 488 raft_consensus.cc:2932] T 2e07f0adca594453bad9433ed0017580 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:45.811161 32691 raft_consensus.cc:2932] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:45.811223 30713 catalog_manager.cc:5484] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index 7 to 8, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:45.812011 4745 ts_tablet_manager.cc:1875] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:45.814396 31134 consensus_queue.cc:237] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:45.815271 32691 raft_consensus.cc:1261] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:45.815403 488 raft_consensus.cc:1261] T 2e07f0adca594453bad9433ed0017580 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:45.815651 4743 consensus_queue.cc:1035] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:45.815973 4743 consensus_queue.cc:1035] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:45.817171 4743 raft_consensus.cc:2932] T 2e07f0adca594453bad9433ed0017580 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:45.817951 488 raft_consensus.cc:2932] T 2e07f0adca594453bad9433ed0017580 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:45.818042 32691 raft_consensus.cc:2932] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:45.817951 4745 ts_tablet_manager.cc:1888] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 14:00:45.818259 4745 log.cc:1196] T aaca0594b68146cdbd18946f123693be P 1fab10dcfb8542fca2b965fed6c67929: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/wals/aaca0594b68146cdbd18946f123693be
I20240502 14:00:45.820621 30707 catalog_manager.cc:5484] T 2e07f0adca594453bad9433ed0017580 P 2716cee98b1b4c4591a40e1f466520e5 reported cstate change: config changed from index 8 to 9, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:45.822548 30701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2e07f0adca594453bad9433ed0017580 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:45.822582 30982 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e07f0adca594453bad9433ed0017580 with delete_type TABLET_DATA_TOMBSTONED (TS 1fab10dcfb8542fca2b965fed6c67929 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:58604
I20240502 14:00:45.822997 4739 raft_consensus.cc:1050] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f: attempting to promote NON_VOTER 3daf9e0837c74a919c89e686597ff56b to VOTER
I20240502 14:00:45.823274 4739 consensus_queue.cc:237] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:45.824522 32691 raft_consensus.cc:1261] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:45.824831 31266 raft_consensus.cc:1261] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:45.824879 4745 tablet_replica.cc:330] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929: stopping tablet replica
I20240502 14:00:45.824985 4745 raft_consensus.cc:2229] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:45.825038 4729 consensus_queue.cc:1035] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:45.825210 4745 raft_consensus.cc:2258] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:45.825479 30700 catalog_manager.cc:4830] TS 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395): tablet aaca0594b68146cdbd18946f123693be (table rebalance_test_table_2 [id=2109014949af4d09893b47c9b10ba7b5]) successfully deleted
I20240502 14:00:45.825649 4729 consensus_queue.cc:1035] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:45.826607 4745 ts_tablet_manager.cc:1875] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:45.826979 488 raft_consensus.cc:1261] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b [term 1 LEARNER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:45.827317 4726 consensus_queue.cc:1035] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:45.827270 4729 raft_consensus.cc:2932] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:45.828441 488 raft_consensus.cc:2932] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:45.829073 32691 raft_consensus.cc:2932] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:45.829258 31266 raft_consensus.cc:2932] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:45.830787 30707 catalog_manager.cc:5484] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index 7 to 8, 3daf9e0837c74a919c89e686597ff56b (127.24.66.71) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:45.833271 4745 ts_tablet_manager.cc:1888] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 14:00:45.833353 4745 log.cc:1196] T 2e07f0adca594453bad9433ed0017580 P 1fab10dcfb8542fca2b965fed6c67929: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/wals/2e07f0adca594453bad9433ed0017580
I20240502 14:00:45.833907 30700 catalog_manager.cc:4830] TS 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395): tablet 2e07f0adca594453bad9433ed0017580 (table rebalance_test_table_8 [id=5cb834cd98904017a28fc7b595efc564]) successfully deleted
I20240502 14:00:45.844724 1430 raft_consensus.cc:1203] T cc64ea7db2064251bed364b800b36b87 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 14:00:45.902153 3834 raft_consensus.cc:1050] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052: attempting to promote NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 to VOTER
I20240502 14:00:45.902526 3834 consensus_queue.cc:237] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } }
I20240502 14:00:45.903807 32691 raft_consensus.cc:1261] T 39239edeafb84b5caaa5dceea06fd4b0 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:45.903940 488 raft_consensus.cc:1261] T 39239edeafb84b5caaa5dceea06fd4b0 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:45.904148 4752 consensus_queue.cc:1035] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:45.904239 31134 raft_consensus.cc:1261] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:45.904430 4754 consensus_queue.cc:1035] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:45.904613 4239 consensus_queue.cc:1035] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:45.906634 4239 raft_consensus.cc:2932] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:45.907148 32691 raft_consensus.cc:2932] T 39239edeafb84b5caaa5dceea06fd4b0 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:45.907229 31134 raft_consensus.cc:2932] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:45.908448 488 raft_consensus.cc:2932] T 39239edeafb84b5caaa5dceea06fd4b0 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:45.909794 30713 catalog_manager.cc:5484] T 39239edeafb84b5caaa5dceea06fd4b0 P 2716cee98b1b4c4591a40e1f466520e5 reported cstate change: config changed from index 7 to 8, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:45.912992 31266 consensus_queue.cc:237] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } }
I20240502 14:00:45.913872 32691 raft_consensus.cc:1261] T 39239edeafb84b5caaa5dceea06fd4b0 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:45.913888 488 raft_consensus.cc:1261] T 39239edeafb84b5caaa5dceea06fd4b0 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:45.914220 4754 consensus_queue.cc:1035] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:45.914386 4752 consensus_queue.cc:1035] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:45.915872 4752 raft_consensus.cc:2932] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:45.917030 32691 raft_consensus.cc:2932] T 39239edeafb84b5caaa5dceea06fd4b0 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:45.917407 30701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 39239edeafb84b5caaa5dceea06fd4b0 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:45.917968 488 raft_consensus.cc:2932] T 39239edeafb84b5caaa5dceea06fd4b0 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:45.918455 30711 catalog_manager.cc:5484] T 39239edeafb84b5caaa5dceea06fd4b0 P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index 8 to 9, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:45.920991 31115 tablet_service.cc:1514] Processing DeleteTablet for tablet 39239edeafb84b5caaa5dceea06fd4b0 with delete_type TABLET_DATA_TOMBSTONED (TS 8c61042437444f5f92c76109dfb229fc not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:42546
I20240502 14:00:45.921294 4763 tablet_replica.cc:330] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc: stopping tablet replica
I20240502 14:00:45.921423 4763 raft_consensus.cc:2229] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:45.921604 4763 raft_consensus.cc:2258] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:45.922135 4763 ts_tablet_manager.cc:1875] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:45.924124 4763 ts_tablet_manager.cc:1888] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 14:00:45.924199 4763 log.cc:1196] T 39239edeafb84b5caaa5dceea06fd4b0 P 8c61042437444f5f92c76109dfb229fc: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/wals/39239edeafb84b5caaa5dceea06fd4b0
I20240502 14:00:45.924666 30701 catalog_manager.cc:4830] TS 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193): tablet 39239edeafb84b5caaa5dceea06fd4b0 (table rebalance_test_table_5 [id=b000c5e9f8984019bef07f52b5bff49d]) successfully deleted
I20240502 14:00:45.942520 31002 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8ec88a191b8d4188985b5e3c91a2a031"
dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:54000
I20240502 14:00:45.942638 31002 raft_consensus.cc:595] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:45.965636 30865 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d0409950632d4eeead8a826034b5a9e4"
dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:52638
I20240502 14:00:45.965773 30865 raft_consensus.cc:595] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:45.973937 32691 raft_consensus.cc:1203] T f571119d6f624fe88bf1effeaf466085 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 14:00:45.980086 4752 raft_consensus.cc:1050] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052: attempting to promote NON_VOTER 2bb442cde32f426bb14791852dd81802 to VOTER
I20240502 14:00:45.980496 4752 consensus_queue.cc:237] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } }
I20240502 14:00:45.981626 31134 raft_consensus.cc:1261] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:45.981673 349 raft_consensus.cc:1261] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:45.981995 619 raft_consensus.cc:1261] T da2f3b3481da476d98c97bf35f055c2e P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:45.982012 3834 consensus_queue.cc:1035] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:45.982167 4754 consensus_queue.cc:1035] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:45.982439 3834 consensus_queue.cc:1035] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:45.984223 4754 raft_consensus.cc:2932] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:45.984673 31134 raft_consensus.cc:2932] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:45.986229 349 raft_consensus.cc:2932] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:45.986697 619 raft_consensus.cc:2932] T da2f3b3481da476d98c97bf35f055c2e P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:45.987241 30714 catalog_manager.cc:5484] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index 7 to 8, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:45.996142 31266 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "da2f3b3481da476d98c97bf35f055c2e"
dest_uuid: "14753c93125941acaad7283edc669052"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:40818
I20240502 14:00:45.996258 31266 raft_consensus.cc:595] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:46.016227 4741 raft_consensus.cc:1050] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc: attempting to promote NON_VOTER 06d9e72a952c44b391752da7674fa6c0 to VOTER
I20240502 14:00:46.016700 4741 consensus_queue.cc:237] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:46.018043 1430 raft_consensus.cc:1261] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:46.018088 619 raft_consensus.cc:1261] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:46.018373 30865 raft_consensus.cc:1261] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:46.018836 4743 consensus_queue.cc:1035] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:46.019160 4255 consensus_queue.cc:1035] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:46.019518 4743 consensus_queue.cc:1035] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:46.021461 4740 raft_consensus.cc:2932] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.021783 1430 raft_consensus.cc:2932] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.021806 619 raft_consensus.cc:2932] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.024163 30713 catalog_manager.cc:5484] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:46.024678 4738 raft_consensus.cc:1050] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f: attempting to promote NON_VOTER 06d9e72a952c44b391752da7674fa6c0 to VOTER
I20240502 14:00:46.025003 4738 consensus_queue.cc:237] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:46.026372 619 raft_consensus.cc:1261] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:46.026696 4726 consensus_queue.cc:1035] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:46.027302 31134 raft_consensus.cc:1261] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:46.027676 4739 consensus_queue.cc:1035] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:46.028257 30865 raft_consensus.cc:2932] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.028362 31134 consensus_queue.cc:237] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:46.029560 619 raft_consensus.cc:1261] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:46.029836 1430 raft_consensus.cc:1261] T 8da28cc4aef043baaa2282f84afd4abf P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:46.030042 1433 raft_consensus.cc:1261] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:46.029973 4743 consensus_queue.cc:1035] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.031564 4739 consensus_queue.cc:1035] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:46.031539 4743 raft_consensus.cc:2932] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.031875 4255 consensus_queue.cc:1035] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.032116 619 raft_consensus.cc:2932] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.033084 30701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 03fb6c61ce4c4ccdafdc9ca7a40db4f5 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:46.033793 30711 catalog_manager.cc:5484] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index 8 to 9, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:46.035003 4739 raft_consensus.cc:2932] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.035156 1430 raft_consensus.cc:2932] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.035960 31134 raft_consensus.cc:2932] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.036842 1433 raft_consensus.cc:2932] T 8da28cc4aef043baaa2282f84afd4abf P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.037525 619 raft_consensus.cc:2932] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.038151 30843 tablet_service.cc:1514] Processing DeleteTablet for tablet 03fb6c61ce4c4ccdafdc9ca7a40db4f5 with delete_type TABLET_DATA_TOMBSTONED (TS ad5befacd63742a1b5b69fdacdb8cf8f not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:45544
I20240502 14:00:46.038625 4871 tablet_replica.cc:330] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f: stopping tablet replica
I20240502 14:00:46.038753 4871 raft_consensus.cc:2229] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:46.038950 4871 raft_consensus.cc:2258] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:46.039743 30711 catalog_manager.cc:5484] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802 reported cstate change: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.042629 4871 ts_tablet_manager.cc:1875] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:46.042958 30865 consensus_queue.cc:237] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:46.043875 619 raft_consensus.cc:1261] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:46.043879 1430 raft_consensus.cc:1261] T 8da28cc4aef043baaa2282f84afd4abf P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:46.044198 4729 consensus_queue.cc:1035] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.044381 4235 consensus_queue.cc:1035] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.045539 4871 ts_tablet_manager.cc:1888] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 14:00:46.045620 4871 log.cc:1196] T 03fb6c61ce4c4ccdafdc9ca7a40db4f5 P ad5befacd63742a1b5b69fdacdb8cf8f: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/wals/03fb6c61ce4c4ccdafdc9ca7a40db4f5
I20240502 14:00:46.046118 30698 catalog_manager.cc:4830] TS ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553): tablet 03fb6c61ce4c4ccdafdc9ca7a40db4f5 (table rebalance_test_table_2 [id=2109014949af4d09893b47c9b10ba7b5]) successfully deleted
I20240502 14:00:46.046345 620 raft_consensus.cc:2932] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.047645 4729 raft_consensus.cc:2932] T 8da28cc4aef043baaa2282f84afd4abf P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.047679 1430 raft_consensus.cc:2932] T 8da28cc4aef043baaa2282f84afd4abf P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.048259 30707 catalog_manager.cc:5484] T 8da28cc4aef043baaa2282f84afd4abf P 2bb442cde32f426bb14791852dd81802 reported cstate change: config changed from index 8 to 9, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.048972 30698 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 8da28cc4aef043baaa2282f84afd4abf with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:46.051084 31115 tablet_service.cc:1514] Processing DeleteTablet for tablet 8da28cc4aef043baaa2282f84afd4abf with delete_type TABLET_DATA_TOMBSTONED (TS 8c61042437444f5f92c76109dfb229fc not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:42546
I20240502 14:00:46.051293 4763 tablet_replica.cc:330] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc: stopping tablet replica
I20240502 14:00:46.051414 4763 raft_consensus.cc:2229] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:46.051578 4763 raft_consensus.cc:2258] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:46.052536 4763 ts_tablet_manager.cc:1875] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:46.054378 4763 ts_tablet_manager.cc:1888] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 14:00:46.054451 4763 log.cc:1196] T 8da28cc4aef043baaa2282f84afd4abf P 8c61042437444f5f92c76109dfb229fc: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/wals/8da28cc4aef043baaa2282f84afd4abf
I20240502 14:00:46.054984 31002 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8ec88a191b8d4188985b5e3c91a2a031"
dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:54018
I20240502 14:00:46.055025 30701 catalog_manager.cc:4830] TS 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193): tablet 8da28cc4aef043baaa2282f84afd4abf (table rebalance_test_table_7 [id=2c4adac4a0a74af9bacbd4987a8a8d39]) successfully deleted
I20240502 14:00:46.055110 31002 raft_consensus.cc:595] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:46.071034 3834 raft_consensus.cc:1050] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052: attempting to promote NON_VOTER 2bb442cde32f426bb14791852dd81802 to VOTER
I20240502 14:00:46.071382 3834 consensus_queue.cc:237] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } }
I20240502 14:00:46.072521 31134 raft_consensus.cc:1261] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:46.072911 4752 consensus_queue.cc:1035] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:46.073053 32691 raft_consensus.cc:1261] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:46.073415 4752 consensus_queue.cc:1035] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:46.073498 620 raft_consensus.cc:1261] T 08588e44069d4044b2d09eb02d252f82 P 2bb442cde32f426bb14791852dd81802 [term 1 LEARNER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:46.074015 4752 consensus_queue.cc:1035] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:46.074668 30865 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d0409950632d4eeead8a826034b5a9e4"
dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:52684
I20240502 14:00:46.074764 30865 raft_consensus.cc:595] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:46.075181 4752 raft_consensus.cc:2932] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:46.076848 32691 raft_consensus.cc:2932] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:46.077229 31134 raft_consensus.cc:2932] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:46.077612 30713 catalog_manager.cc:5484] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index 7 to 8, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:46.080269 620 raft_consensus.cc:2932] T 08588e44069d4044b2d09eb02d252f82 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2bb442cde32f426bb14791852dd81802 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:46.083196 31266 consensus_queue.cc:237] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } }
I20240502 14:00:46.084165 620 raft_consensus.cc:1261] T 08588e44069d4044b2d09eb02d252f82 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:46.084354 32691 raft_consensus.cc:1261] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:46.084717 4239 consensus_queue.cc:1035] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.085057 4754 consensus_queue.cc:1035] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.086187 4239 raft_consensus.cc:2932] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:46.086889 620 raft_consensus.cc:2932] T 08588e44069d4044b2d09eb02d252f82 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:46.088456 30701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 08588e44069d4044b2d09eb02d252f82 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:46.088657 30714 catalog_manager.cc:5484] T 08588e44069d4044b2d09eb02d252f82 P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index 8 to 9, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:46.088915 32691 raft_consensus.cc:2932] T 08588e44069d4044b2d09eb02d252f82 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:46.091012 31115 tablet_service.cc:1514] Processing DeleteTablet for tablet 08588e44069d4044b2d09eb02d252f82 with delete_type TABLET_DATA_TOMBSTONED (TS 8c61042437444f5f92c76109dfb229fc not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:42546
I20240502 14:00:46.091207 4763 tablet_replica.cc:330] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc: stopping tablet replica
I20240502 14:00:46.091327 4763 raft_consensus.cc:2229] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:46.091471 4763 raft_consensus.cc:2258] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:46.093137 4763 ts_tablet_manager.cc:1875] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:46.093813 4692 raft_consensus.cc:979] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929: : Instructing follower 06d9e72a952c44b391752da7674fa6c0 to start an election
I20240502 14:00:46.093907 4681 raft_consensus.cc:1067] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Signalling peer 06d9e72a952c44b391752da7674fa6c0 to start an election
I20240502 14:00:46.094393 1430 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8ec88a191b8d4188985b5e3c91a2a031"
dest_uuid: "06d9e72a952c44b391752da7674fa6c0"
from {username='slave'} at 127.24.66.66:52877
I20240502 14:00:46.094568 1430 raft_consensus.cc:482] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:00:46.094667 1430 raft_consensus.cc:3037] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:46.095208 4763 ts_tablet_manager.cc:1888] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 14:00:46.095266 4763 log.cc:1196] T 08588e44069d4044b2d09eb02d252f82 P 8c61042437444f5f92c76109dfb229fc: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/wals/08588e44069d4044b2d09eb02d252f82
I20240502 14:00:46.095829 1430 raft_consensus.cc:504] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:46.096735 4741 raft_consensus.cc:1050] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc: attempting to promote NON_VOTER f769c13ece0b40179ba120dbebbaba98 to VOTER
I20240502 14:00:46.096467 30701 catalog_manager.cc:4830] TS 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193): tablet 08588e44069d4044b2d09eb02d252f82 (table rebalance_test_table_1 [id=a4f4cd48a9854811903166c61a300083]) successfully deleted
I20240502 14:00:46.097100 4741 consensus_queue.cc:237] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } }
I20240502 14:00:46.097463 1430 leader_election.cc:288] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0 [CANDIDATE]: Term 2 election: Requested vote from peers 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395), 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193), 3daf9e0837c74a919c89e686597ff56b (127.24.66.71:35161)
I20240502 14:00:46.098474 30865 raft_consensus.cc:1261] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:46.098672 31134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ec88a191b8d4188985b5e3c91a2a031" candidate_uuid: "06d9e72a952c44b391752da7674fa6c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "8c61042437444f5f92c76109dfb229fc"
I20240502 14:00:46.098773 31134 raft_consensus.cc:3037] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:46.098748 31002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ec88a191b8d4188985b5e3c91a2a031" candidate_uuid: "06d9e72a952c44b391752da7674fa6c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
I20240502 14:00:46.098855 31002 raft_consensus.cc:3032] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:00:46.098901 31002 raft_consensus.cc:729] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Running, Role: LEADER
I20240502 14:00:46.099040 31002 consensus_queue.cc:260] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:46.099200 31002 raft_consensus.cc:3037] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:46.099511 31134 raft_consensus.cc:2454] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d9e72a952c44b391752da7674fa6c0 in term 2.
I20240502 14:00:46.100003 349 raft_consensus.cc:1261] T 66e2378d5c0f400299a170df8a939577 P f769c13ece0b40179ba120dbebbaba98 [term 1 LEARNER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:46.100075 31002 raft_consensus.cc:2454] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d9e72a952c44b391752da7674fa6c0 in term 2.
I20240502 14:00:46.100432 4740 consensus_queue.cc:1035] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:46.100620 4255 consensus_queue.cc:1035] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:46.100961 1379 leader_election.cc:302] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 06d9e72a952c44b391752da7674fa6c0, 1fab10dcfb8542fca2b965fed6c67929, 8c61042437444f5f92c76109dfb229fc; no voters:
I20240502 14:00:46.101187 4547 raft_consensus.cc:2781] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:00:46.101287 620 raft_consensus.cc:1261] T 66e2378d5c0f400299a170df8a939577 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:46.101347 4547 raft_consensus.cc:686] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0 [term 2 LEADER]: Becoming Leader. State: Replica: 06d9e72a952c44b391752da7674fa6c0, State: Running, Role: LEADER
I20240502 14:00:46.101634 4547 consensus_queue.cc:237] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:46.102372 4740 consensus_queue.cc:1035] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:46.104066 4743 raft_consensus.cc:2932] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:46.104364 30707 catalog_manager.cc:5484] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0 reported cstate change: term changed from 1 to 2, leader changed from 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) to 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73). New cstate: current_term: 2 leader_uuid: "06d9e72a952c44b391752da7674fa6c0" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:46.104689 30865 raft_consensus.cc:2932] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:46.106029 620 raft_consensus.cc:2932] T 66e2378d5c0f400299a170df8a939577 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:46.106904 30714 catalog_manager.cc:5484] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index 7 to 8, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:46.106917 349 raft_consensus.cc:2932] T 66e2378d5c0f400299a170df8a939577 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, f769c13ece0b40179ba120dbebbaba98 (127.24.66.70) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:46.111439 31134 consensus_queue.cc:237] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } }
I20240502 14:00:46.113168 349 raft_consensus.cc:1261] T 66e2378d5c0f400299a170df8a939577 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:46.113569 4255 consensus_queue.cc:1035] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.113621 620 raft_consensus.cc:1261] T 66e2378d5c0f400299a170df8a939577 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:46.114027 4255 consensus_queue.cc:1035] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.115020 4255 raft_consensus.cc:2932] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:46.115348 349 raft_consensus.cc:2932] T 66e2378d5c0f400299a170df8a939577 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:46.115794 620 raft_consensus.cc:2932] T 66e2378d5c0f400299a170df8a939577 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } }
I20240502 14:00:46.116928 30701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 66e2378d5c0f400299a170df8a939577 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:46.117672 30714 catalog_manager.cc:5484] T 66e2378d5c0f400299a170df8a939577 P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index 8 to 9, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:46.119789 30843 tablet_service.cc:1514] Processing DeleteTablet for tablet 66e2378d5c0f400299a170df8a939577 with delete_type TABLET_DATA_TOMBSTONED (TS ad5befacd63742a1b5b69fdacdb8cf8f not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:45544
I20240502 14:00:46.119963 4871 tablet_replica.cc:330] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f: stopping tablet replica
I20240502 14:00:46.120083 4871 raft_consensus.cc:2229] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:46.120257 4871 raft_consensus.cc:2258] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:46.120985 488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ec88a191b8d4188985b5e3c91a2a031" candidate_uuid: "06d9e72a952c44b391752da7674fa6c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "3daf9e0837c74a919c89e686597ff56b"
I20240502 14:00:46.121068 4871 ts_tablet_manager.cc:1875] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:46.121099 488 raft_consensus.cc:3037] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:46.121917 488 raft_consensus.cc:2454] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d9e72a952c44b391752da7674fa6c0 in term 2.
I20240502 14:00:46.122586 4871 ts_tablet_manager.cc:1888] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 14:00:46.122632 4871 log.cc:1196] T 66e2378d5c0f400299a170df8a939577 P ad5befacd63742a1b5b69fdacdb8cf8f: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/wals/66e2378d5c0f400299a170df8a939577
I20240502 14:00:46.123039 30698 catalog_manager.cc:4830] TS ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553): tablet 66e2378d5c0f400299a170df8a939577 (table rebalance_test_table_4 [id=4cd0d0b0efb44a2c966dd894b6e48601]) successfully deleted
I20240502 14:00:46.127305 31266 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "da2f3b3481da476d98c97bf35f055c2e"
dest_uuid: "14753c93125941acaad7283edc669052"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:40858
I20240502 14:00:46.127404 31266 raft_consensus.cc:595] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:46.147444 31002 consensus_queue.cc:237] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:46.148756 349 raft_consensus.cc:1261] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240502 14:00:46.148772 30935 consensus_peers.cc:485] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 -> Peer 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73:34725): Couldn't send request to peer 06d9e72a952c44b391752da7674fa6c0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1f3459ebf25f45dd810046420f5dcd81. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:46.148828 31134 raft_consensus.cc:1261] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:46.149096 4681 consensus_queue.cc:1035] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:46.149263 4692 consensus_queue.cc:1035] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:46.150820 4681 raft_consensus.cc:2932] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:46.152859 31134 raft_consensus.cc:2932] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:46.153332 30713 catalog_manager.cc:5484] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:46.153995 349 raft_consensus.cc:2932] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:46.192042 30865 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d0409950632d4eeead8a826034b5a9e4"
dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:52728
I20240502 14:00:46.192148 30865 raft_consensus.cc:595] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:46.205929 4741 raft_consensus.cc:1050] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc: attempting to promote NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 to VOTER
I20240502 14:00:46.206313 4741 consensus_queue.cc:237] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 2.8, Last appended by leader: 5, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } }
I20240502 14:00:46.207696 32691 raft_consensus.cc:1261] T 2e02dc2a98d04bddafdd1b933d89b4dd P 2716cee98b1b4c4591a40e1f466520e5 [term 2 LEARNER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 14:00:46.207755 30865 raft_consensus.cc:1261] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f [term 2 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 14:00:46.207751 488 raft_consensus.cc:1261] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b [term 2 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 2 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 14:00:46.208110 4743 consensus_queue.cc:1035] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.208498 4740 consensus_queue.cc:1035] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.208654 4743 consensus_queue.cc:1035] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.210216 4743 raft_consensus.cc:2932] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [term 2 LEADER]: Committing config change with OpId 2.9: config changed from index 8 to 9, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:46.210680 32691 raft_consensus.cc:2932] T 2e02dc2a98d04bddafdd1b933d89b4dd P 2716cee98b1b4c4591a40e1f466520e5 [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 8 to 9, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:46.210705 488 raft_consensus.cc:2932] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 8 to 9, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:46.212182 30865 raft_consensus.cc:2932] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f [term 2 FOLLOWER]: Committing config change with OpId 2.9: config changed from index 8 to 9, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:46.213060 30714 catalog_manager.cc:5484] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b reported cstate change: config changed from index 8 to 9, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:46.215253 31266 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "da2f3b3481da476d98c97bf35f055c2e"
dest_uuid: "14753c93125941acaad7283edc669052"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:40888
I20240502 14:00:46.215382 31266 raft_consensus.cc:595] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:46.216650 31134 consensus_queue.cc:237] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } }
I20240502 14:00:46.217824 32691 raft_consensus.cc:1261] T 2e02dc2a98d04bddafdd1b933d89b4dd P 2716cee98b1b4c4591a40e1f466520e5 [term 2 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240502 14:00:46.217824 488 raft_consensus.cc:1261] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b [term 2 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240502 14:00:46.218371 4741 consensus_queue.cc:1035] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:00:46.218559 4740 consensus_queue.cc:1035] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:00:46.221455 4741 raft_consensus.cc:2932] T 2e02dc2a98d04bddafdd1b933d89b4dd P 8c61042437444f5f92c76109dfb229fc [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:46.221855 32691 raft_consensus.cc:2932] T 2e02dc2a98d04bddafdd1b933d89b4dd P 2716cee98b1b4c4591a40e1f466520e5 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:46.221855 488 raft_consensus.cc:2932] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 9 to 10, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:46.223028 30701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2e02dc2a98d04bddafdd1b933d89b4dd with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:46.224862 30713 catalog_manager.cc:5484] T 2e02dc2a98d04bddafdd1b933d89b4dd P 3daf9e0837c74a919c89e686597ff56b reported cstate change: config changed from index 9 to 10, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New cstate: current_term: 2 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:46.227064 4727 raft_consensus.cc:979] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f: : Instructing follower 3daf9e0837c74a919c89e686597ff56b to start an election
I20240502 14:00:46.227164 4727 raft_consensus.cc:1067] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Signalling peer 3daf9e0837c74a919c89e686597ff56b to start an election
I20240502 14:00:46.227715 30843 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e02dc2a98d04bddafdd1b933d89b4dd with delete_type TABLET_DATA_TOMBSTONED (TS ad5befacd63742a1b5b69fdacdb8cf8f not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:45544
I20240502 14:00:46.227687 488 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d0409950632d4eeead8a826034b5a9e4"
dest_uuid: "3daf9e0837c74a919c89e686597ff56b"
from {username='slave'} at 127.24.66.65:40197
I20240502 14:00:46.227842 488 raft_consensus.cc:482] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:00:46.227891 488 raft_consensus.cc:3037] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:46.227891 4871 tablet_replica.cc:330] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f: stopping tablet replica
I20240502 14:00:46.228041 4871 raft_consensus.cc:2229] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:46.228192 4871 raft_consensus.cc:2258] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:46.228783 488 raft_consensus.cc:504] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:46.229858 488 leader_election.cc:288] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b [CANDIDATE]: Term 2 election: Requested vote from peers ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553), 14753c93125941acaad7283edc669052 (127.24.66.68:37029), 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69:35027)
I20240502 14:00:46.230141 31266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0409950632d4eeead8a826034b5a9e4" candidate_uuid: "3daf9e0837c74a919c89e686597ff56b" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "14753c93125941acaad7283edc669052"
I20240502 14:00:46.230271 31266 raft_consensus.cc:3037] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:46.230690 30865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0409950632d4eeead8a826034b5a9e4" candidate_uuid: "3daf9e0837c74a919c89e686597ff56b" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
I20240502 14:00:46.230774 30865 raft_consensus.cc:3032] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:00:46.230805 30865 raft_consensus.cc:729] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Running, Role: LEADER
I20240502 14:00:46.230937 30865 consensus_queue.cc:260] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:46.231109 30865 raft_consensus.cc:3037] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:46.231261 31266 raft_consensus.cc:2454] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3daf9e0837c74a919c89e686597ff56b in term 2.
I20240502 14:00:46.232048 30865 raft_consensus.cc:2454] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3daf9e0837c74a919c89e686597ff56b in term 2.
I20240502 14:00:46.232167 4871 ts_tablet_manager.cc:1875] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:46.232314 420 leader_election.cc:302] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 3daf9e0837c74a919c89e686597ff56b, ad5befacd63742a1b5b69fdacdb8cf8f; no voters:
I20240502 14:00:46.232720 4519 raft_consensus.cc:2781] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:00:46.232878 4519 raft_consensus.cc:686] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b [term 2 LEADER]: Becoming Leader. State: Replica: 3daf9e0837c74a919c89e686597ff56b, State: Running, Role: LEADER
I20240502 14:00:46.233173 4519 consensus_queue.cc:237] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:46.234197 4871 ts_tablet_manager.cc:1888] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240502 14:00:46.234256 4871 log.cc:1196] T 2e02dc2a98d04bddafdd1b933d89b4dd P ad5befacd63742a1b5b69fdacdb8cf8f: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/wals/2e02dc2a98d04bddafdd1b933d89b4dd
I20240502 14:00:46.234692 30698 catalog_manager.cc:4830] TS ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553): tablet 2e02dc2a98d04bddafdd1b933d89b4dd (table rebalance_test_table_0 [id=a5535db3201345e78ac8467a4d987c08]) successfully deleted
I20240502 14:00:46.235924 30707 catalog_manager.cc:5484] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b reported cstate change: term changed from 1 to 2, leader changed from ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) to 3daf9e0837c74a919c89e686597ff56b (127.24.66.71). New cstate: current_term: 2 leader_uuid: "3daf9e0837c74a919c89e686597ff56b" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:46.237658 30865 consensus_queue.cc:237] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:46.238940 32691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0409950632d4eeead8a826034b5a9e4" candidate_uuid: "3daf9e0837c74a919c89e686597ff56b" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "2716cee98b1b4c4591a40e1f466520e5"
I20240502 14:00:46.238991 349 raft_consensus.cc:1261] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:46.239038 31266 raft_consensus.cc:1261] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:46.239078 32691 raft_consensus.cc:3037] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Advancing to term 2
W20240502 14:00:46.239221 30797 consensus_peers.cc:485] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f -> Peer 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73:34725): Couldn't send request to peer 06d9e72a952c44b391752da7674fa6c0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5a3c9c93437c43b1a9c0326f2880a04e. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:46.239396 4738 consensus_queue.cc:1035] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:46.239527 4726 consensus_queue.cc:1035] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:46.239974 32691 raft_consensus.cc:2454] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3daf9e0837c74a919c89e686597ff56b in term 2.
I20240502 14:00:46.241189 4739 raft_consensus.cc:2932] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:46.241660 31266 raft_consensus.cc:2932] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:46.243240 349 raft_consensus.cc:2932] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:46.243353 30707 catalog_manager.cc:5484] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:46.295390 31266 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "da2f3b3481da476d98c97bf35f055c2e"
dest_uuid: "14753c93125941acaad7283edc669052"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:40918
I20240502 14:00:46.295490 31266 raft_consensus.cc:595] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:46.309528 4743 raft_consensus.cc:1050] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc: attempting to promote NON_VOTER 06d9e72a952c44b391752da7674fa6c0 to VOTER
I20240502 14:00:46.309991 4743 consensus_queue.cc:237] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:46.311326 1430 raft_consensus.cc:1261] T cc64ea7db2064251bed364b800b36b87 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:46.311336 620 raft_consensus.cc:1261] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:46.311398 31266 raft_consensus.cc:1261] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:46.311724 4255 consensus_queue.cc:1035] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:46.311862 4741 consensus_queue.cc:1035] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:46.311978 4740 consensus_queue.cc:1035] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:46.314409 4743 raft_consensus.cc:2932] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.314889 31266 raft_consensus.cc:2932] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.314890 620 raft_consensus.cc:2932] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.316701 1430 raft_consensus.cc:2932] T cc64ea7db2064251bed364b800b36b87 P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.317593 30714 catalog_manager.cc:5484] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.320936 31134 consensus_queue.cc:237] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:46.321836 620 raft_consensus.cc:1261] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:46.322083 1430 raft_consensus.cc:1261] T cc64ea7db2064251bed364b800b36b87 P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:46.322172 4255 consensus_queue.cc:1035] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.322436 4741 consensus_queue.cc:1035] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.323488 4255 raft_consensus.cc:2932] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.324283 1430 raft_consensus.cc:2932] T cc64ea7db2064251bed364b800b36b87 P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.324705 620 raft_consensus.cc:2932] T cc64ea7db2064251bed364b800b36b87 P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.325444 30701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet cc64ea7db2064251bed364b800b36b87 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:46.326483 30713 catalog_manager.cc:5484] T cc64ea7db2064251bed364b800b36b87 P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index 8 to 9, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:46.328895 31246 tablet_service.cc:1514] Processing DeleteTablet for tablet cc64ea7db2064251bed364b800b36b87 with delete_type TABLET_DATA_TOMBSTONED (TS 14753c93125941acaad7283edc669052 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:53826
I20240502 14:00:46.329208 5081 tablet_replica.cc:330] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052: stopping tablet replica
I20240502 14:00:46.329311 5081 raft_consensus.cc:2229] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:46.329444 5081 raft_consensus.cc:2258] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:46.329946 5081 ts_tablet_manager.cc:1875] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:46.331715 5081 ts_tablet_manager.cc:1888] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 14:00:46.331770 5081 log.cc:1196] T cc64ea7db2064251bed364b800b36b87 P 14753c93125941acaad7283edc669052: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/cc64ea7db2064251bed364b800b36b87
I20240502 14:00:46.332198 30701 catalog_manager.cc:4830] TS 14753c93125941acaad7283edc669052 (127.24.66.68:37029): tablet cc64ea7db2064251bed364b800b36b87 (table rebalance_test_table_5 [id=b000c5e9f8984019bef07f52b5bff49d]) successfully deleted
I20240502 14:00:46.347110 4727 raft_consensus.cc:1050] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f: attempting to promote NON_VOTER 06d9e72a952c44b391752da7674fa6c0 to VOTER
I20240502 14:00:46.347421 4727 consensus_queue.cc:237] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:46.348711 31134 raft_consensus.cc:1261] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:46.348757 32691 raft_consensus.cc:1261] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:46.348994 1430 raft_consensus.cc:1261] T efd8e3b804cf41bebf5e629b65571979 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:46.349038 4727 consensus_queue.cc:1035] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:46.349197 4739 consensus_queue.cc:1035] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:46.349349 4738 consensus_queue.cc:1035] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:46.351011 4727 raft_consensus.cc:2932] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.351454 31134 raft_consensus.cc:2932] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.352806 32691 raft_consensus.cc:2932] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.352985 1430 raft_consensus.cc:2932] T efd8e3b804cf41bebf5e629b65571979 P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.353662 30707 catalog_manager.cc:5484] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.357481 30865 consensus_queue.cc:237] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:46.358599 32690 raft_consensus.cc:1261] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:46.358958 1430 raft_consensus.cc:1261] T efd8e3b804cf41bebf5e629b65571979 P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:46.359318 4738 consensus_queue.cc:1035] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.359525 4726 consensus_queue.cc:1035] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.360361 4752 raft_consensus.cc:1050] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052: attempting to promote NON_VOTER 2716cee98b1b4c4591a40e1f466520e5 to VOTER
I20240502 14:00:46.360702 4752 consensus_queue.cc:237] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } }
I20240502 14:00:46.360932 4738 raft_consensus.cc:2932] T efd8e3b804cf41bebf5e629b65571979 P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.361433 32691 raft_consensus.cc:2932] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.362028 31134 raft_consensus.cc:1261] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:46.362180 488 raft_consensus.cc:1261] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:46.362524 3834 consensus_queue.cc:1035] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:46.362818 32690 raft_consensus.cc:1261] T f571119d6f624fe88bf1effeaf466085 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 LEARNER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:46.363019 4239 consensus_queue.cc:1035] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:46.363325 4239 consensus_queue.cc:1035] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:46.363893 1430 raft_consensus.cc:2932] T efd8e3b804cf41bebf5e629b65571979 P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.364214 30698 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet efd8e3b804cf41bebf5e629b65571979 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:46.365072 30713 catalog_manager.cc:5484] T efd8e3b804cf41bebf5e629b65571979 P 2716cee98b1b4c4591a40e1f466520e5 reported cstate change: config changed from index 8 to 9, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:46.366140 4754 raft_consensus.cc:2932] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:46.367098 488 raft_consensus.cc:2932] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:46.367960 31134 raft_consensus.cc:2932] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:46.368069 32691 raft_consensus.cc:2932] T f571119d6f624fe88bf1effeaf466085 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:46.368615 30714 catalog_manager.cc:5484] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index 7 to 8, 2716cee98b1b4c4591a40e1f466520e5 (127.24.66.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:46.371027 31115 tablet_service.cc:1514] Processing DeleteTablet for tablet efd8e3b804cf41bebf5e629b65571979 with delete_type TABLET_DATA_TOMBSTONED (TS 8c61042437444f5f92c76109dfb229fc not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:42546
I20240502 14:00:46.371287 4763 tablet_replica.cc:330] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc: stopping tablet replica
I20240502 14:00:46.371423 4763 raft_consensus.cc:2229] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:46.371553 4763 raft_consensus.cc:2258] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:46.372359 4763 ts_tablet_manager.cc:1875] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:46.373394 31266 consensus_queue.cc:237] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } }
I20240502 14:00:46.374325 4763 ts_tablet_manager.cc:1888] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 14:00:46.374404 4763 log.cc:1196] T efd8e3b804cf41bebf5e629b65571979 P 8c61042437444f5f92c76109dfb229fc: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/wals/efd8e3b804cf41bebf5e629b65571979
I20240502 14:00:46.374486 32690 raft_consensus.cc:1261] T f571119d6f624fe88bf1effeaf466085 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:46.374576 488 raft_consensus.cc:1261] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:46.374843 3834 consensus_queue.cc:1035] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.375111 4239 consensus_queue.cc:1035] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.375718 30701 catalog_manager.cc:4830] TS 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193): tablet efd8e3b804cf41bebf5e629b65571979 (table rebalance_test_table_4 [id=4cd0d0b0efb44a2c966dd894b6e48601]) successfully deleted
I20240502 14:00:46.377193 4754 raft_consensus.cc:2932] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:46.377506 487 raft_consensus.cc:2932] T f571119d6f624fe88bf1effeaf466085 P 3daf9e0837c74a919c89e686597ff56b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:46.378381 32690 raft_consensus.cc:2932] T f571119d6f624fe88bf1effeaf466085 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } }
I20240502 14:00:46.379416 30707 catalog_manager.cc:5484] T f571119d6f624fe88bf1effeaf466085 P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index 8 to 9, VOTER 8c61042437444f5f92c76109dfb229fc (127.24.66.67) evicted. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:46.379782 30701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f571119d6f624fe88bf1effeaf466085 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:46.380815 3834 raft_consensus.cc:979] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052: : Instructing follower 8c61042437444f5f92c76109dfb229fc to start an election
I20240502 14:00:46.380898 3834 raft_consensus.cc:1067] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Signalling peer 8c61042437444f5f92c76109dfb229fc to start an election
I20240502 14:00:46.381353 31134 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "da2f3b3481da476d98c97bf35f055c2e"
dest_uuid: "8c61042437444f5f92c76109dfb229fc"
from {username='slave'} at 127.24.66.68:38355
I20240502 14:00:46.381455 31134 raft_consensus.cc:482] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:00:46.381531 31134 raft_consensus.cc:3037] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:46.382212 31134 raft_consensus.cc:504] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } }
I20240502 14:00:46.382539 31134 leader_election.cc:288] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 2 election: Requested vote from peers 14753c93125941acaad7283edc669052 (127.24.66.68:37029), f769c13ece0b40179ba120dbebbaba98 (127.24.66.70:38291), 2bb442cde32f426bb14791852dd81802 (127.24.66.72:34475)
I20240502 14:00:46.382934 31266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da2f3b3481da476d98c97bf35f055c2e" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "14753c93125941acaad7283edc669052"
I20240502 14:00:46.382963 620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da2f3b3481da476d98c97bf35f055c2e" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "2bb442cde32f426bb14791852dd81802"
I20240502 14:00:46.382982 349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da2f3b3481da476d98c97bf35f055c2e" candidate_uuid: "8c61042437444f5f92c76109dfb229fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "f769c13ece0b40179ba120dbebbaba98"
I20240502 14:00:46.383049 31266 raft_consensus.cc:3032] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:00:46.383088 31266 raft_consensus.cc:729] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 14753c93125941acaad7283edc669052, State: Running, Role: LEADER
I20240502 14:00:46.383087 349 raft_consensus.cc:3037] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:46.383112 620 raft_consensus.cc:3037] T da2f3b3481da476d98c97bf35f055c2e P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:46.383224 31266 consensus_queue.cc:260] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } }
I20240502 14:00:46.383375 31266 raft_consensus.cc:3037] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:46.383882 620 raft_consensus.cc:2454] T da2f3b3481da476d98c97bf35f055c2e P 2bb442cde32f426bb14791852dd81802 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 2.
I20240502 14:00:46.384027 349 raft_consensus.cc:2454] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 2.
I20240502 14:00:46.384137 31266 raft_consensus.cc:2454] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c61042437444f5f92c76109dfb229fc in term 2.
I20240502 14:00:46.384708 31069 leader_election.cc:302] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 14753c93125941acaad7283edc669052, 8c61042437444f5f92c76109dfb229fc, f769c13ece0b40179ba120dbebbaba98; no voters:
I20240502 14:00:46.384910 4741 raft_consensus.cc:2781] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:00:46.384915 31115 tablet_service.cc:1514] Processing DeleteTablet for tablet f571119d6f624fe88bf1effeaf466085 with delete_type TABLET_DATA_TOMBSTONED (TS 8c61042437444f5f92c76109dfb229fc not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:42546
I20240502 14:00:46.385007 4741 raft_consensus.cc:686] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [term 2 LEADER]: Becoming Leader. State: Replica: 8c61042437444f5f92c76109dfb229fc, State: Running, Role: LEADER
I20240502 14:00:46.385125 4741 consensus_queue.cc:237] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } }
I20240502 14:00:46.386073 4763 tablet_replica.cc:330] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc: stopping tablet replica
I20240502 14:00:46.386204 4763 raft_consensus.cc:2229] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:46.386369 4763 raft_consensus.cc:2258] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:46.386947 4763 ts_tablet_manager.cc:1875] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:46.387101 30711 catalog_manager.cc:5484] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc reported cstate change: term changed from 1 to 2, leader changed from 14753c93125941acaad7283edc669052 (127.24.66.68) to 8c61042437444f5f92c76109dfb229fc (127.24.66.67). New cstate: current_term: 2 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:46.388736 4763 ts_tablet_manager.cc:1888] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 14:00:46.388809 4763 log.cc:1196] T f571119d6f624fe88bf1effeaf466085 P 8c61042437444f5f92c76109dfb229fc: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/wals/f571119d6f624fe88bf1effeaf466085
I20240502 14:00:46.389281 30701 catalog_manager.cc:4830] TS 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193): tablet f571119d6f624fe88bf1effeaf466085 (table rebalance_test_table_3 [id=3fb9e83c8b7c4d90a375d14e792f650d]) successfully deleted
I20240502 14:00:46.517418 31002 raft_consensus.cc:1261] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 2 FOLLOWER]: Refusing update from remote peer 06d9e72a952c44b391752da7674fa6c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 14:00:46.517969 4919 consensus_queue.cc:1035] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.537609 487 raft_consensus.cc:1261] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b [term 2 FOLLOWER]: Refusing update from remote peer 06d9e72a952c44b391752da7674fa6c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 14:00:46.538201 4919 consensus_queue.cc:1035] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.540849 31134 raft_consensus.cc:1261] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc [term 2 FOLLOWER]: Refusing update from remote peer 06d9e72a952c44b391752da7674fa6c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 14:00:46.542258 4919 consensus_queue.cc:1035] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.543705 5135 ts_tablet_manager.cc:902] T 1f3459ebf25f45dd810046420f5dcd81 P 06d9e72a952c44b391752da7674fa6c0: Initiating tablet copy from peer 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:46.544448 5135 tablet_copy_client.cc:308] T 1f3459ebf25f45dd810046420f5dcd81 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.66:36395
I20240502 14:00:46.544862 31023 tablet_copy_service.cc:140] P 1fab10dcfb8542fca2b965fed6c67929: Received BeginTabletCopySession request for tablet 1f3459ebf25f45dd810046420f5dcd81 from peer 06d9e72a952c44b391752da7674fa6c0 ({username='slave'} at 127.24.66.73:41497)
I20240502 14:00:46.544958 31023 tablet_copy_service.cc:161] P 1fab10dcfb8542fca2b965fed6c67929: Beginning new tablet copy session on tablet 1f3459ebf25f45dd810046420f5dcd81 from peer 06d9e72a952c44b391752da7674fa6c0 at {username='slave'} at 127.24.66.73:41497: session id = 06d9e72a952c44b391752da7674fa6c0-1f3459ebf25f45dd810046420f5dcd81
I20240502 14:00:46.545855 31023 tablet_copy_source_session.cc:205] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:46.546859 5135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f3459ebf25f45dd810046420f5dcd81. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:46.547180 31002 consensus_queue.cc:237] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
W20240502 14:00:46.548743 30935 consensus_peers.cc:485] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 -> Peer 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73:34725): Couldn't send request to peer 06d9e72a952c44b391752da7674fa6c0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: be25ac1c6e7242519ff636242718cb79. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:46.548763 349 raft_consensus.cc:1261] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:46.548851 5135 tablet_copy_client.cc:783] T 1f3459ebf25f45dd810046420f5dcd81 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:46.548981 5135 tablet_copy_client.cc:647] T 1f3459ebf25f45dd810046420f5dcd81 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:46.549221 31266 raft_consensus.cc:1261] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:46.549273 4692 consensus_queue.cc:1035] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:46.549613 4692 consensus_queue.cc:1035] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:46.550212 5144 tablet_copy_client.cc:1002] T 1f3459ebf25f45dd810046420f5dcd81 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:46.550897 4682 raft_consensus.cc:2932] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:46.551576 31266 raft_consensus.cc:2932] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:46.552825 349 raft_consensus.cc:2932] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:46.553223 30713 catalog_manager.cc:5484] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:46.560869 31266 consensus_queue.cc:237] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:46.562207 32690 raft_consensus.cc:1261] T 954fa01ea4d04842b0218b33de347aa1 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:46.562454 31002 raft_consensus.cc:1261] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:46.562559 4754 consensus_queue.cc:1035] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:46.562762 3834 consensus_queue.cc:1035] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:46.564291 4754 raft_consensus.cc:2932] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:46.564707 31002 raft_consensus.cc:2932] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:46.566586 30711 catalog_manager.cc:5484] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:46.567144 32690 raft_consensus.cc:2932] T 954fa01ea4d04842b0218b33de347aa1 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
W20240502 14:00:46.568259 31201 consensus_peers.cc:485] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 -> Peer 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73:34725): Couldn't send request to peer 06d9e72a952c44b391752da7674fa6c0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 954fa01ea4d04842b0218b33de347aa1. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:46.694670 30865 raft_consensus.cc:1261] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 2 FOLLOWER]: Refusing update from remote peer 3daf9e0837c74a919c89e686597ff56b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 14:00:46.695109 5028 consensus_queue.cc:1035] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.703804 32690 raft_consensus.cc:1261] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5 [term 2 FOLLOWER]: Refusing update from remote peer 3daf9e0837c74a919c89e686597ff56b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 14:00:46.704164 5028 consensus_queue.cc:1035] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.706996 31266 raft_consensus.cc:1261] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052 [term 2 FOLLOWER]: Refusing update from remote peer 3daf9e0837c74a919c89e686597ff56b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 14:00:46.707616 5192 consensus_queue.cc:1035] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.784788 349 raft_consensus.cc:1261] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98 [term 2 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 14:00:46.785085 4741 consensus_queue.cc:1035] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.793092 5196 ts_tablet_manager.cc:902] T 5a3c9c93437c43b1a9c0326f2880a04e P 06d9e72a952c44b391752da7674fa6c0: Initiating tablet copy from peer ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553)
I20240502 14:00:46.793521 5196 tablet_copy_client.cc:308] T 5a3c9c93437c43b1a9c0326f2880a04e P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.65:39553
I20240502 14:00:46.793936 30887 tablet_copy_service.cc:140] P ad5befacd63742a1b5b69fdacdb8cf8f: Received BeginTabletCopySession request for tablet 5a3c9c93437c43b1a9c0326f2880a04e from peer 06d9e72a952c44b391752da7674fa6c0 ({username='slave'} at 127.24.66.73:39479)
I20240502 14:00:46.794032 30887 tablet_copy_service.cc:161] P ad5befacd63742a1b5b69fdacdb8cf8f: Beginning new tablet copy session on tablet 5a3c9c93437c43b1a9c0326f2880a04e from peer 06d9e72a952c44b391752da7674fa6c0 at {username='slave'} at 127.24.66.73:39479: session id = 06d9e72a952c44b391752da7674fa6c0-5a3c9c93437c43b1a9c0326f2880a04e
I20240502 14:00:46.794867 30887 tablet_copy_source_session.cc:205] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:46.795650 5196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a3c9c93437c43b1a9c0326f2880a04e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:46.797396 5196 tablet_copy_client.cc:783] T 5a3c9c93437c43b1a9c0326f2880a04e P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:46.797531 5196 tablet_copy_client.cc:647] T 5a3c9c93437c43b1a9c0326f2880a04e P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:46.798439 5200 tablet_copy_client.cc:1002] T 5a3c9c93437c43b1a9c0326f2880a04e P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:46.818214 620 raft_consensus.cc:1261] T da2f3b3481da476d98c97bf35f055c2e P 2bb442cde32f426bb14791852dd81802 [term 2 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 14:00:46.818598 4741 consensus_queue.cc:1035] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.823026 31266 raft_consensus.cc:1261] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 2 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 14:00:46.823423 4741 consensus_queue.cc:1035] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:46.825903 31134 consensus_queue.cc:237] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } }
I20240502 14:00:46.827003 620 raft_consensus.cc:1261] T da2f3b3481da476d98c97bf35f055c2e P 2bb442cde32f426bb14791852dd81802 [term 2 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240502 14:00:46.827064 349 raft_consensus.cc:1261] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98 [term 2 FOLLOWER]: Refusing update from remote peer 8c61042437444f5f92c76109dfb229fc: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240502 14:00:46.827505 4741 consensus_queue.cc:1035] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:00:46.828929 4740 consensus_queue.cc:1035] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:00:46.829254 4740 raft_consensus.cc:2932] T da2f3b3481da476d98c97bf35f055c2e P 8c61042437444f5f92c76109dfb229fc [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:46.829725 620 raft_consensus.cc:2932] T da2f3b3481da476d98c97bf35f055c2e P 2bb442cde32f426bb14791852dd81802 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:46.831817 30701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet da2f3b3481da476d98c97bf35f055c2e with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:46.832060 30713 catalog_manager.cc:5484] T da2f3b3481da476d98c97bf35f055c2e P 2bb442cde32f426bb14791852dd81802 reported cstate change: config changed from index 8 to 10, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New cstate: current_term: 2 leader_uuid: "8c61042437444f5f92c76109dfb229fc" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:46.832901 349 raft_consensus.cc:2932] T da2f3b3481da476d98c97bf35f055c2e P f769c13ece0b40179ba120dbebbaba98 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } }
I20240502 14:00:46.834849 31246 tablet_service.cc:1514] Processing DeleteTablet for tablet da2f3b3481da476d98c97bf35f055c2e with delete_type TABLET_DATA_TOMBSTONED (TS 14753c93125941acaad7283edc669052 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:53826
I20240502 14:00:46.835158 5228 tablet_replica.cc:330] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052: stopping tablet replica
I20240502 14:00:46.835395 5228 raft_consensus.cc:2229] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:46.835615 5228 raft_consensus.cc:2258] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:46.836910 5228 ts_tablet_manager.cc:1875] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:46.838707 5228 ts_tablet_manager.cc:1888] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240502 14:00:46.838771 5228 log.cc:1196] T da2f3b3481da476d98c97bf35f055c2e P 14753c93125941acaad7283edc669052: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/da2f3b3481da476d98c97bf35f055c2e
I20240502 14:00:46.839213 30701 catalog_manager.cc:4830] TS 14753c93125941acaad7283edc669052 (127.24.66.68:37029): tablet da2f3b3481da476d98c97bf35f055c2e (table rebalance_test_table_6 [id=99c461286f6a4e4f81cc54ae6fc11b21]) successfully deleted
I20240502 14:00:47.005293 5269 ts_tablet_manager.cc:902] T 954fa01ea4d04842b0218b33de347aa1 P 06d9e72a952c44b391752da7674fa6c0: Initiating tablet copy from peer 14753c93125941acaad7283edc669052 (127.24.66.68:37029)
I20240502 14:00:47.005805 5269 tablet_copy_client.cc:308] T 954fa01ea4d04842b0218b33de347aa1 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:37029
I20240502 14:00:47.008986 31288 tablet_copy_service.cc:140] P 14753c93125941acaad7283edc669052: Received BeginTabletCopySession request for tablet 954fa01ea4d04842b0218b33de347aa1 from peer 06d9e72a952c44b391752da7674fa6c0 ({username='slave'} at 127.24.66.73:33875)
I20240502 14:00:47.009068 31288 tablet_copy_service.cc:161] P 14753c93125941acaad7283edc669052: Beginning new tablet copy session on tablet 954fa01ea4d04842b0218b33de347aa1 from peer 06d9e72a952c44b391752da7674fa6c0 at {username='slave'} at 127.24.66.73:33875: session id = 06d9e72a952c44b391752da7674fa6c0-954fa01ea4d04842b0218b33de347aa1
I20240502 14:00:47.009734 31288 tablet_copy_source_session.cc:205] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:47.010497 5269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 954fa01ea4d04842b0218b33de347aa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:47.012194 5269 tablet_copy_client.cc:783] T 954fa01ea4d04842b0218b33de347aa1 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:47.012351 5269 tablet_copy_client.cc:647] T 954fa01ea4d04842b0218b33de347aa1 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:47.013233 5273 tablet_copy_client.cc:1002] T 954fa01ea4d04842b0218b33de347aa1 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:47.064101 5275 ts_tablet_manager.cc:902] T be25ac1c6e7242519ff636242718cb79 P 06d9e72a952c44b391752da7674fa6c0: Initiating tablet copy from peer 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:47.064517 5275 tablet_copy_client.cc:308] T be25ac1c6e7242519ff636242718cb79 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.66:36395
I20240502 14:00:47.064953 31023 tablet_copy_service.cc:140] P 1fab10dcfb8542fca2b965fed6c67929: Received BeginTabletCopySession request for tablet be25ac1c6e7242519ff636242718cb79 from peer 06d9e72a952c44b391752da7674fa6c0 ({username='slave'} at 127.24.66.73:41497)
I20240502 14:00:47.065039 31023 tablet_copy_service.cc:161] P 1fab10dcfb8542fca2b965fed6c67929: Beginning new tablet copy session on tablet be25ac1c6e7242519ff636242718cb79 from peer 06d9e72a952c44b391752da7674fa6c0 at {username='slave'} at 127.24.66.73:41497: session id = 06d9e72a952c44b391752da7674fa6c0-be25ac1c6e7242519ff636242718cb79
I20240502 14:00:47.065690 31023 tablet_copy_source_session.cc:205] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:47.066349 5275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be25ac1c6e7242519ff636242718cb79. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:47.067824 5275 tablet_copy_client.cc:783] T be25ac1c6e7242519ff636242718cb79 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:47.067950 5275 tablet_copy_client.cc:647] T be25ac1c6e7242519ff636242718cb79 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:47.068837 5279 tablet_copy_client.cc:1002] T be25ac1c6e7242519ff636242718cb79 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:47.100827 487 consensus_queue.cc:237] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } }
I20240502 14:00:47.101853 31266 raft_consensus.cc:1261] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052 [term 2 FOLLOWER]: Refusing update from remote peer 3daf9e0837c74a919c89e686597ff56b: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240502 14:00:47.101867 32690 raft_consensus.cc:1261] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5 [term 2 FOLLOWER]: Refusing update from remote peer 3daf9e0837c74a919c89e686597ff56b: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240502 14:00:47.102314 5028 consensus_queue.cc:1035] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:00:47.102725 5192 consensus_queue.cc:1035] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:00:47.104566 4519 raft_consensus.cc:2932] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:47.104969 32690 raft_consensus.cc:2932] T d0409950632d4eeead8a826034b5a9e4 P 2716cee98b1b4c4591a40e1f466520e5 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:47.106123 30700 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet d0409950632d4eeead8a826034b5a9e4 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:47.107348 30713 catalog_manager.cc:5484] T d0409950632d4eeead8a826034b5a9e4 P 3daf9e0837c74a919c89e686597ff56b reported cstate change: config changed from index 8 to 10, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New cstate: current_term: 2 leader_uuid: "3daf9e0837c74a919c89e686597ff56b" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:47.109661 31266 raft_consensus.cc:2932] T d0409950632d4eeead8a826034b5a9e4 P 14753c93125941acaad7283edc669052 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } }
I20240502 14:00:47.110924 30843 tablet_service.cc:1514] Processing DeleteTablet for tablet d0409950632d4eeead8a826034b5a9e4 with delete_type TABLET_DATA_TOMBSTONED (TS ad5befacd63742a1b5b69fdacdb8cf8f not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:45544
I20240502 14:00:47.111994 5301 tablet_replica.cc:330] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f: stopping tablet replica
I20240502 14:00:47.112169 5301 raft_consensus.cc:2229] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:47.112327 5301 raft_consensus.cc:2258] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:47.113245 5301 ts_tablet_manager.cc:1875] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:47.115748 5301 ts_tablet_manager.cc:1888] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240502 14:00:47.115869 5301 log.cc:1196] T d0409950632d4eeead8a826034b5a9e4 P ad5befacd63742a1b5b69fdacdb8cf8f: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/wals/d0409950632d4eeead8a826034b5a9e4
I20240502 14:00:47.116436 30698 catalog_manager.cc:4830] TS ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553): tablet d0409950632d4eeead8a826034b5a9e4 (table rebalance_test_table_7 [id=2c4adac4a0a74af9bacbd4987a8a8d39]) successfully deleted
I20240502 14:00:47.119880 1430 consensus_queue.cc:237] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:47.121333 487 raft_consensus.cc:1261] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b [term 2 FOLLOWER]: Refusing update from remote peer 06d9e72a952c44b391752da7674fa6c0: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240502 14:00:47.121394 31134 raft_consensus.cc:1261] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc [term 2 FOLLOWER]: Refusing update from remote peer 06d9e72a952c44b391752da7674fa6c0: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240502 14:00:47.121721 4547 consensus_queue.cc:1035] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:00:47.122042 5302 consensus_queue.cc:1035] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:00:47.123456 4547 raft_consensus.cc:2932] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0 [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:47.123874 31134 raft_consensus.cc:2932] T 8ec88a191b8d4188985b5e3c91a2a031 P 8c61042437444f5f92c76109dfb229fc [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:47.125254 30700 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 8ec88a191b8d4188985b5e3c91a2a031 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:47.126400 30713 catalog_manager.cc:5484] T 8ec88a191b8d4188985b5e3c91a2a031 P 06d9e72a952c44b391752da7674fa6c0 reported cstate change: config changed from index 8 to 10, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New cstate: current_term: 2 leader_uuid: "06d9e72a952c44b391752da7674fa6c0" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:47.127287 487 raft_consensus.cc:2932] T 8ec88a191b8d4188985b5e3c91a2a031 P 3daf9e0837c74a919c89e686597ff56b [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "3daf9e0837c74a919c89e686597ff56b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 35161 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:47.128751 30982 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ec88a191b8d4188985b5e3c91a2a031 with delete_type TABLET_DATA_TOMBSTONED (TS 1fab10dcfb8542fca2b965fed6c67929 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:58604
I20240502 14:00:47.129626 5315 tablet_replica.cc:330] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929: stopping tablet replica
I20240502 14:00:47.129820 5315 raft_consensus.cc:2229] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:47.129981 5315 raft_consensus.cc:2258] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:47.130563 5315 ts_tablet_manager.cc:1875] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:47.132504 5315 ts_tablet_manager.cc:1888] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240502 14:00:47.132565 5315 log.cc:1196] T 8ec88a191b8d4188985b5e3c91a2a031 P 1fab10dcfb8542fca2b965fed6c67929: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/wals/8ec88a191b8d4188985b5e3c91a2a031
I20240502 14:00:47.132975 30700 catalog_manager.cc:4830] TS 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395): tablet 8ec88a191b8d4188985b5e3c91a2a031 (table rebalance_test_table_1 [id=a4f4cd48a9854811903166c61a300083]) successfully deleted
I20240502 14:00:47.163941 31001 consensus_queue.cc:237] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:47.165177 620 raft_consensus.cc:1261] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:00:47.165186 31266 raft_consensus.cc:1261] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20240502 14:00:47.165253 30935 consensus_peers.cc:485] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 -> Peer 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73:34725): Couldn't send request to peer 06d9e72a952c44b391752da7674fa6c0. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8c77a03ce6a142debba75e0d387aa8ec. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:00:47.165529 5336 consensus_queue.cc:1035] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:47.165692 5337 consensus_queue.cc:1035] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:00:47.167258 5337 raft_consensus.cc:2932] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:47.167604 31266 raft_consensus.cc:2932] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:47.169883 620 raft_consensus.cc:2932] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:47.170002 30707 catalog_manager.cc:5484] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index 6 to 7, NON_VOTER 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) added. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } } }
I20240502 14:00:47.771122 5433 ts_tablet_manager.cc:902] T 8c77a03ce6a142debba75e0d387aa8ec P 06d9e72a952c44b391752da7674fa6c0: Initiating tablet copy from peer 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395)
I20240502 14:00:47.771556 5433 tablet_copy_client.cc:308] T 8c77a03ce6a142debba75e0d387aa8ec P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.66:36395
I20240502 14:00:47.771919 31023 tablet_copy_service.cc:140] P 1fab10dcfb8542fca2b965fed6c67929: Received BeginTabletCopySession request for tablet 8c77a03ce6a142debba75e0d387aa8ec from peer 06d9e72a952c44b391752da7674fa6c0 ({username='slave'} at 127.24.66.73:41497)
I20240502 14:00:47.771996 31023 tablet_copy_service.cc:161] P 1fab10dcfb8542fca2b965fed6c67929: Beginning new tablet copy session on tablet 8c77a03ce6a142debba75e0d387aa8ec from peer 06d9e72a952c44b391752da7674fa6c0 at {username='slave'} at 127.24.66.73:41497: session id = 06d9e72a952c44b391752da7674fa6c0-8c77a03ce6a142debba75e0d387aa8ec
I20240502 14:00:47.772645 31023 tablet_copy_source_session.cc:205] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:00:47.773384 5433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c77a03ce6a142debba75e0d387aa8ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:47.775185 5433 tablet_copy_client.cc:783] T 8c77a03ce6a142debba75e0d387aa8ec P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Starting download of 0 data blocks...
I20240502 14:00:47.775331 5433 tablet_copy_client.cc:647] T 8c77a03ce6a142debba75e0d387aa8ec P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Starting download of 1 WAL segments...
I20240502 14:00:47.776278 5437 tablet_copy_client.cc:1002] T 8c77a03ce6a142debba75e0d387aa8ec P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Injecting latency into file download: 1500
I20240502 14:00:48.050690 5135 tablet_copy_client.cc:523] T 1f3459ebf25f45dd810046420f5dcd81 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:48.051951 5135 tablet_bootstrap.cc:492] T 1f3459ebf25f45dd810046420f5dcd81 P 06d9e72a952c44b391752da7674fa6c0: Bootstrap starting.
I20240502 14:00:48.069061 5135 tablet_bootstrap.cc:492] T 1f3459ebf25f45dd810046420f5dcd81 P 06d9e72a952c44b391752da7674fa6c0: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:48.069279 5135 tablet_bootstrap.cc:492] T 1f3459ebf25f45dd810046420f5dcd81 P 06d9e72a952c44b391752da7674fa6c0: Bootstrap complete.
I20240502 14:00:48.069453 5135 ts_tablet_manager.cc:1367] T 1f3459ebf25f45dd810046420f5dcd81 P 06d9e72a952c44b391752da7674fa6c0: Time spent bootstrapping tablet: real 0.018s user 0.018s sys 0.000s
I20240502 14:00:48.069819 5135 raft_consensus.cc:348] T 1f3459ebf25f45dd810046420f5dcd81 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:48.069931 5135 raft_consensus.cc:729] T 1f3459ebf25f45dd810046420f5dcd81 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 06d9e72a952c44b391752da7674fa6c0, State: Initialized, Role: LEARNER
I20240502 14:00:48.070006 5135 consensus_queue.cc:260] T 1f3459ebf25f45dd810046420f5dcd81 P 06d9e72a952c44b391752da7674fa6c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:48.070219 5135 ts_tablet_manager.cc:1398] T 1f3459ebf25f45dd810046420f5dcd81 P 06d9e72a952c44b391752da7674fa6c0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:48.070655 31023 tablet_copy_service.cc:342] P 1fab10dcfb8542fca2b965fed6c67929: Request end of tablet copy session 06d9e72a952c44b391752da7674fa6c0-1f3459ebf25f45dd810046420f5dcd81 received from {username='slave'} at 127.24.66.73:41497
I20240502 14:00:48.070748 31023 tablet_copy_service.cc:433] P 1fab10dcfb8542fca2b965fed6c67929: ending tablet copy session 06d9e72a952c44b391752da7674fa6c0-1f3459ebf25f45dd810046420f5dcd81 on tablet 1f3459ebf25f45dd810046420f5dcd81 with peer 06d9e72a952c44b391752da7674fa6c0
I20240502 14:00:48.153193 1430 raft_consensus.cc:1203] T 1f3459ebf25f45dd810046420f5dcd81 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 14:00:48.298730 5196 tablet_copy_client.cc:523] T 5a3c9c93437c43b1a9c0326f2880a04e P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:48.300155 5196 tablet_bootstrap.cc:492] T 5a3c9c93437c43b1a9c0326f2880a04e P 06d9e72a952c44b391752da7674fa6c0: Bootstrap starting.
I20240502 14:00:48.324916 5196 tablet_bootstrap.cc:492] T 5a3c9c93437c43b1a9c0326f2880a04e P 06d9e72a952c44b391752da7674fa6c0: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:48.325241 5196 tablet_bootstrap.cc:492] T 5a3c9c93437c43b1a9c0326f2880a04e P 06d9e72a952c44b391752da7674fa6c0: Bootstrap complete.
I20240502 14:00:48.325489 5196 ts_tablet_manager.cc:1367] T 5a3c9c93437c43b1a9c0326f2880a04e P 06d9e72a952c44b391752da7674fa6c0: Time spent bootstrapping tablet: real 0.025s user 0.021s sys 0.005s
I20240502 14:00:48.325860 5196 raft_consensus.cc:348] T 5a3c9c93437c43b1a9c0326f2880a04e P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:48.325959 5196 raft_consensus.cc:729] T 5a3c9c93437c43b1a9c0326f2880a04e P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 06d9e72a952c44b391752da7674fa6c0, State: Initialized, Role: LEARNER
I20240502 14:00:48.326037 5196 consensus_queue.cc:260] T 5a3c9c93437c43b1a9c0326f2880a04e P 06d9e72a952c44b391752da7674fa6c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:48.326303 5196 ts_tablet_manager.cc:1398] T 5a3c9c93437c43b1a9c0326f2880a04e P 06d9e72a952c44b391752da7674fa6c0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:48.326833 30887 tablet_copy_service.cc:342] P ad5befacd63742a1b5b69fdacdb8cf8f: Request end of tablet copy session 06d9e72a952c44b391752da7674fa6c0-5a3c9c93437c43b1a9c0326f2880a04e received from {username='slave'} at 127.24.66.73:39479
I20240502 14:00:48.326915 30887 tablet_copy_service.cc:433] P ad5befacd63742a1b5b69fdacdb8cf8f: ending tablet copy session 06d9e72a952c44b391752da7674fa6c0-5a3c9c93437c43b1a9c0326f2880a04e on tablet 5a3c9c93437c43b1a9c0326f2880a04e with peer 06d9e72a952c44b391752da7674fa6c0
I20240502 14:00:48.513501 5269 tablet_copy_client.cc:523] T 954fa01ea4d04842b0218b33de347aa1 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:48.514786 5269 tablet_bootstrap.cc:492] T 954fa01ea4d04842b0218b33de347aa1 P 06d9e72a952c44b391752da7674fa6c0: Bootstrap starting.
I20240502 14:00:48.533104 5269 tablet_bootstrap.cc:492] T 954fa01ea4d04842b0218b33de347aa1 P 06d9e72a952c44b391752da7674fa6c0: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:48.533382 5269 tablet_bootstrap.cc:492] T 954fa01ea4d04842b0218b33de347aa1 P 06d9e72a952c44b391752da7674fa6c0: Bootstrap complete.
I20240502 14:00:48.533553 5269 ts_tablet_manager.cc:1367] T 954fa01ea4d04842b0218b33de347aa1 P 06d9e72a952c44b391752da7674fa6c0: Time spent bootstrapping tablet: real 0.019s user 0.019s sys 0.000s
I20240502 14:00:48.533967 5269 raft_consensus.cc:348] T 954fa01ea4d04842b0218b33de347aa1 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:48.534102 5269 raft_consensus.cc:729] T 954fa01ea4d04842b0218b33de347aa1 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 06d9e72a952c44b391752da7674fa6c0, State: Initialized, Role: LEARNER
I20240502 14:00:48.534229 5269 consensus_queue.cc:260] T 954fa01ea4d04842b0218b33de347aa1 P 06d9e72a952c44b391752da7674fa6c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:48.534534 5269 ts_tablet_manager.cc:1398] T 954fa01ea4d04842b0218b33de347aa1 P 06d9e72a952c44b391752da7674fa6c0: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:00:48.535382 31288 tablet_copy_service.cc:342] P 14753c93125941acaad7283edc669052: Request end of tablet copy session 06d9e72a952c44b391752da7674fa6c0-954fa01ea4d04842b0218b33de347aa1 received from {username='slave'} at 127.24.66.73:33875
I20240502 14:00:48.535470 31288 tablet_copy_service.cc:433] P 14753c93125941acaad7283edc669052: ending tablet copy session 06d9e72a952c44b391752da7674fa6c0-954fa01ea4d04842b0218b33de347aa1 on tablet 954fa01ea4d04842b0218b33de347aa1 with peer 06d9e72a952c44b391752da7674fa6c0
I20240502 14:00:48.569054 5275 tablet_copy_client.cc:523] T be25ac1c6e7242519ff636242718cb79 P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:48.570282 5275 tablet_bootstrap.cc:492] T be25ac1c6e7242519ff636242718cb79 P 06d9e72a952c44b391752da7674fa6c0: Bootstrap starting.
I20240502 14:00:48.576169 1433 raft_consensus.cc:1203] T 954fa01ea4d04842b0218b33de347aa1 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 14:00:48.587169 5275 tablet_bootstrap.cc:492] T be25ac1c6e7242519ff636242718cb79 P 06d9e72a952c44b391752da7674fa6c0: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:48.587380 5275 tablet_bootstrap.cc:492] T be25ac1c6e7242519ff636242718cb79 P 06d9e72a952c44b391752da7674fa6c0: Bootstrap complete.
I20240502 14:00:48.587498 5275 ts_tablet_manager.cc:1367] T be25ac1c6e7242519ff636242718cb79 P 06d9e72a952c44b391752da7674fa6c0: Time spent bootstrapping tablet: real 0.017s user 0.015s sys 0.000s
I20240502 14:00:48.587792 5275 raft_consensus.cc:348] T be25ac1c6e7242519ff636242718cb79 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:48.587884 5275 raft_consensus.cc:729] T be25ac1c6e7242519ff636242718cb79 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 06d9e72a952c44b391752da7674fa6c0, State: Initialized, Role: LEARNER
I20240502 14:00:48.587953 5275 consensus_queue.cc:260] T be25ac1c6e7242519ff636242718cb79 P 06d9e72a952c44b391752da7674fa6c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:48.588172 5275 ts_tablet_manager.cc:1398] T be25ac1c6e7242519ff636242718cb79 P 06d9e72a952c44b391752da7674fa6c0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:48.588647 31023 tablet_copy_service.cc:342] P 1fab10dcfb8542fca2b965fed6c67929: Request end of tablet copy session 06d9e72a952c44b391752da7674fa6c0-be25ac1c6e7242519ff636242718cb79 received from {username='slave'} at 127.24.66.73:41497
I20240502 14:00:48.588732 31023 tablet_copy_service.cc:433] P 1fab10dcfb8542fca2b965fed6c67929: ending tablet copy session 06d9e72a952c44b391752da7674fa6c0-be25ac1c6e7242519ff636242718cb79 on tablet be25ac1c6e7242519ff636242718cb79 with peer 06d9e72a952c44b391752da7674fa6c0
I20240502 14:00:48.594533 1433 raft_consensus.cc:1203] T 5a3c9c93437c43b1a9c0326f2880a04e P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 14:00:48.638546 5337 raft_consensus.cc:1050] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929: attempting to promote NON_VOTER 06d9e72a952c44b391752da7674fa6c0 to VOTER
I20240502 14:00:48.638968 5337 consensus_queue.cc:237] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:48.640266 1433 raft_consensus.cc:1261] T 1f3459ebf25f45dd810046420f5dcd81 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:48.640419 31134 raft_consensus.cc:1261] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:48.640413 354 raft_consensus.cc:1261] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:48.640808 4692 consensus_queue.cc:1035] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:48.640960 5548 consensus_queue.cc:1035] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:48.641162 5431 consensus_queue.cc:1035] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:48.643697 5431 raft_consensus.cc:2932] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:48.644227 1433 raft_consensus.cc:2932] T 1f3459ebf25f45dd810046420f5dcd81 P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:48.645143 354 raft_consensus.cc:2932] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:48.646196 31134 raft_consensus.cc:2932] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:48.647114 30713 catalog_manager.cc:5484] T 1f3459ebf25f45dd810046420f5dcd81 P 06d9e72a952c44b391752da7674fa6c0 reported cstate change: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:48.787024 31001 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1f3459ebf25f45dd810046420f5dcd81"
dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:54348
I20240502 14:00:48.787137 31001 raft_consensus.cc:595] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:49.026723 5337 raft_consensus.cc:979] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929: : Instructing follower f769c13ece0b40179ba120dbebbaba98 to start an election
I20240502 14:00:49.026834 5337 raft_consensus.cc:1067] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Signalling peer f769c13ece0b40179ba120dbebbaba98 to start an election
I20240502 14:00:49.028755 354 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1f3459ebf25f45dd810046420f5dcd81"
dest_uuid: "f769c13ece0b40179ba120dbebbaba98"
from {username='slave'} at 127.24.66.66:47469
I20240502 14:00:49.028962 354 raft_consensus.cc:482] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:00:49.029039 354 raft_consensus.cc:3037] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:49.030097 354 raft_consensus.cc:504] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:49.031446 354 leader_election.cc:288] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [CANDIDATE]: Term 2 election: Requested vote from peers 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395), 8c61042437444f5f92c76109dfb229fc (127.24.66.67:34193), 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73:34725)
I20240502 14:00:49.031894 31001 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1f3459ebf25f45dd810046420f5dcd81"
dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:54390
I20240502 14:00:49.032021 31001 raft_consensus.cc:595] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:49.032140 31134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f3459ebf25f45dd810046420f5dcd81" candidate_uuid: "f769c13ece0b40179ba120dbebbaba98" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "8c61042437444f5f92c76109dfb229fc"
I20240502 14:00:49.032294 31134 raft_consensus.cc:3037] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:49.033421 31134 raft_consensus.cc:2454] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f769c13ece0b40179ba120dbebbaba98 in term 2.
I20240502 14:00:49.033658 31002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f3459ebf25f45dd810046420f5dcd81" candidate_uuid: "f769c13ece0b40179ba120dbebbaba98" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "1fab10dcfb8542fca2b965fed6c67929"
I20240502 14:00:49.033766 31002 raft_consensus.cc:3032] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:00:49.033804 31002 raft_consensus.cc:729] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1fab10dcfb8542fca2b965fed6c67929, State: Running, Role: LEADER
I20240502 14:00:49.033939 31002 consensus_queue.cc:260] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:49.034124 31002 raft_consensus.cc:3037] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:49.035014 31002 raft_consensus.cc:2454] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f769c13ece0b40179ba120dbebbaba98 in term 2.
I20240502 14:00:49.035605 32757 leader_election.cc:302] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1fab10dcfb8542fca2b965fed6c67929, 8c61042437444f5f92c76109dfb229fc, f769c13ece0b40179ba120dbebbaba98; no voters:
I20240502 14:00:49.035812 5552 raft_consensus.cc:2781] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:00:49.035993 5552 raft_consensus.cc:686] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [term 2 LEADER]: Becoming Leader. State: Replica: f769c13ece0b40179ba120dbebbaba98, State: Running, Role: LEADER
I20240502 14:00:49.036275 5552 consensus_queue.cc:237] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:49.037539 1433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f3459ebf25f45dd810046420f5dcd81" candidate_uuid: "f769c13ece0b40179ba120dbebbaba98" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "06d9e72a952c44b391752da7674fa6c0"
I20240502 14:00:49.037753 1433 raft_consensus.cc:3037] T 1f3459ebf25f45dd810046420f5dcd81 P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:49.038714 1433 raft_consensus.cc:2454] T 1f3459ebf25f45dd810046420f5dcd81 P 06d9e72a952c44b391752da7674fa6c0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f769c13ece0b40179ba120dbebbaba98 in term 2.
I20240502 14:00:49.039052 30707 catalog_manager.cc:5484] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 reported cstate change: term changed from 1 to 2, leader changed from 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) to f769c13ece0b40179ba120dbebbaba98 (127.24.66.70). New cstate: current_term: 2 leader_uuid: "f769c13ece0b40179ba120dbebbaba98" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:49.056504 1433 raft_consensus.cc:1203] T be25ac1c6e7242519ff636242718cb79 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 14:00:49.093757 5621 raft_consensus.cc:1050] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052: attempting to promote NON_VOTER 06d9e72a952c44b391752da7674fa6c0 to VOTER
I20240502 14:00:49.094154 5621 consensus_queue.cc:237] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:49.095346 32690 raft_consensus.cc:1261] T 954fa01ea4d04842b0218b33de347aa1 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:49.095583 31002 raft_consensus.cc:1261] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:49.095670 1433 raft_consensus.cc:1261] T 954fa01ea4d04842b0218b33de347aa1 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:49.095744 5587 consensus_queue.cc:1035] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:49.096230 5587 consensus_queue.cc:1035] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:49.097391 5587 consensus_queue.cc:1035] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:49.099025 5621 raft_consensus.cc:2932] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:49.099491 1433 raft_consensus.cc:2932] T 954fa01ea4d04842b0218b33de347aa1 P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:49.100833 32690 raft_consensus.cc:2932] T 954fa01ea4d04842b0218b33de347aa1 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:49.101627 30707 catalog_manager.cc:5484] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:49.103708 31002 raft_consensus.cc:2932] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:49.105089 31266 consensus_queue.cc:237] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:49.106123 32690 raft_consensus.cc:1261] T 954fa01ea4d04842b0218b33de347aa1 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:49.106926 5623 consensus_queue.cc:1035] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:49.107205 1433 raft_consensus.cc:1261] T 954fa01ea4d04842b0218b33de347aa1 P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:49.107625 5623 consensus_queue.cc:1035] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:49.108875 5587 raft_consensus.cc:2932] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:49.109639 32690 raft_consensus.cc:2932] T 954fa01ea4d04842b0218b33de347aa1 P 2716cee98b1b4c4591a40e1f466520e5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:49.110244 30701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 954fa01ea4d04842b0218b33de347aa1 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:49.110746 1433 raft_consensus.cc:2932] T 954fa01ea4d04842b0218b33de347aa1 P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:49.111196 30711 catalog_manager.cc:5484] T 954fa01ea4d04842b0218b33de347aa1 P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index 8 to 9, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New cstate: current_term: 1 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2716cee98b1b4c4591a40e1f466520e5" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 35027 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:49.113690 30982 tablet_service.cc:1514] Processing DeleteTablet for tablet 954fa01ea4d04842b0218b33de347aa1 with delete_type TABLET_DATA_TOMBSTONED (TS 1fab10dcfb8542fca2b965fed6c67929 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:58604
I20240502 14:00:49.114049 5628 tablet_replica.cc:330] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929: stopping tablet replica
I20240502 14:00:49.114213 5628 raft_consensus.cc:2229] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:49.114451 5628 raft_consensus.cc:2258] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:49.115072 5628 ts_tablet_manager.cc:1875] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:49.116801 5628 ts_tablet_manager.cc:1888] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 14:00:49.116859 5628 log.cc:1196] T 954fa01ea4d04842b0218b33de347aa1 P 1fab10dcfb8542fca2b965fed6c67929: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/wals/954fa01ea4d04842b0218b33de347aa1
I20240502 14:00:49.117331 30700 catalog_manager.cc:4830] TS 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395): tablet 954fa01ea4d04842b0218b33de347aa1 (table rebalance_test_table_6 [id=99c461286f6a4e4f81cc54ae6fc11b21]) successfully deleted
I20240502 14:00:49.159127 5630 raft_consensus.cc:1050] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f: attempting to promote NON_VOTER 06d9e72a952c44b391752da7674fa6c0 to VOTER
I20240502 14:00:49.159538 5630 consensus_queue.cc:237] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:49.160741 31266 raft_consensus.cc:1261] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:49.160864 1433 raft_consensus.cc:1261] T 5a3c9c93437c43b1a9c0326f2880a04e P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:49.160864 354 raft_consensus.cc:1261] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Refusing update from remote peer ad5befacd63742a1b5b69fdacdb8cf8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:49.161149 5632 consensus_queue.cc:1035] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:49.161319 5630 consensus_queue.cc:1035] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:49.161464 5629 consensus_queue.cc:1035] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:49.163317 5630 raft_consensus.cc:2932] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:49.163802 31266 raft_consensus.cc:2932] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:49.164425 354 raft_consensus.cc:2932] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:49.166016 30707 catalog_manager.cc:5484] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f reported cstate change: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:49.166227 1433 raft_consensus.cc:2932] T 5a3c9c93437c43b1a9c0326f2880a04e P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:49.259210 30865 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5a3c9c93437c43b1a9c0326f2880a04e"
dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:52902
I20240502 14:00:49.259341 30865 raft_consensus.cc:595] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:49.276507 5433 tablet_copy_client.cc:523] T 8c77a03ce6a142debba75e0d387aa8ec P 06d9e72a952c44b391752da7674fa6c0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:00:49.277756 5433 tablet_bootstrap.cc:492] T 8c77a03ce6a142debba75e0d387aa8ec P 06d9e72a952c44b391752da7674fa6c0: Bootstrap starting.
I20240502 14:00:49.291165 30865 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5a3c9c93437c43b1a9c0326f2880a04e"
dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:52928
I20240502 14:00:49.291285 30865 raft_consensus.cc:595] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:49.300882 5433 tablet_bootstrap.cc:492] T 8c77a03ce6a142debba75e0d387aa8ec P 06d9e72a952c44b391752da7674fa6c0: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:00:49.320520 5433 tablet_bootstrap.cc:492] T 8c77a03ce6a142debba75e0d387aa8ec P 06d9e72a952c44b391752da7674fa6c0: Bootstrap complete.
I20240502 14:00:49.320703 5433 ts_tablet_manager.cc:1367] T 8c77a03ce6a142debba75e0d387aa8ec P 06d9e72a952c44b391752da7674fa6c0: Time spent bootstrapping tablet: real 0.043s user 0.023s sys 0.000s
I20240502 14:00:49.321085 5433 raft_consensus.cc:348] T 8c77a03ce6a142debba75e0d387aa8ec P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:49.321183 5433 raft_consensus.cc:729] T 8c77a03ce6a142debba75e0d387aa8ec P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 06d9e72a952c44b391752da7674fa6c0, State: Initialized, Role: LEARNER
I20240502 14:00:49.321275 5433 consensus_queue.cc:260] T 8c77a03ce6a142debba75e0d387aa8ec P 06d9e72a952c44b391752da7674fa6c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: true } }
I20240502 14:00:49.321556 5433 ts_tablet_manager.cc:1398] T 8c77a03ce6a142debba75e0d387aa8ec P 06d9e72a952c44b391752da7674fa6c0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:00:49.322016 31023 tablet_copy_service.cc:342] P 1fab10dcfb8542fca2b965fed6c67929: Request end of tablet copy session 06d9e72a952c44b391752da7674fa6c0-8c77a03ce6a142debba75e0d387aa8ec received from {username='slave'} at 127.24.66.73:41497
I20240502 14:00:49.322091 31023 tablet_copy_service.cc:433] P 1fab10dcfb8542fca2b965fed6c67929: ending tablet copy session 06d9e72a952c44b391752da7674fa6c0-8c77a03ce6a142debba75e0d387aa8ec on tablet 8c77a03ce6a142debba75e0d387aa8ec with peer 06d9e72a952c44b391752da7674fa6c0
I20240502 14:00:49.494904 31002 raft_consensus.cc:1261] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 2 FOLLOWER]: Refusing update from remote peer f769c13ece0b40179ba120dbebbaba98: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 14:00:49.495355 5552 consensus_queue.cc:1035] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:49.517019 30865 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5a3c9c93437c43b1a9c0326f2880a04e"
dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:52948
I20240502 14:00:49.517127 30865 raft_consensus.cc:595] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Received request to transfer leadership
I20240502 14:00:49.534443 1433 raft_consensus.cc:1261] T 1f3459ebf25f45dd810046420f5dcd81 P 06d9e72a952c44b391752da7674fa6c0 [term 2 FOLLOWER]: Refusing update from remote peer f769c13ece0b40179ba120dbebbaba98: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 14:00:49.534767 5552 consensus_queue.cc:1035] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:49.537451 31134 raft_consensus.cc:1261] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc [term 2 FOLLOWER]: Refusing update from remote peer f769c13ece0b40179ba120dbebbaba98: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 14:00:49.537878 5720 consensus_queue.cc:1035] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:49.632530 5725 raft_consensus.cc:1050] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929: attempting to promote NON_VOTER 06d9e72a952c44b391752da7674fa6c0 to VOTER
I20240502 14:00:49.632928 5725 consensus_queue.cc:237] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:49.634150 1433 raft_consensus.cc:1261] T be25ac1c6e7242519ff636242718cb79 P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:49.634485 354 raft_consensus.cc:1261] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:49.634670 5725 consensus_queue.cc:1035] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:49.634913 31266 raft_consensus.cc:1261] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:49.634985 5727 consensus_queue.cc:1035] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:49.635314 5725 consensus_queue.cc:1035] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:49.640849 5727 raft_consensus.cc:2932] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:49.641788 354 raft_consensus.cc:2932] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:49.642699 1433 raft_consensus.cc:2932] T be25ac1c6e7242519ff636242718cb79 P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:49.644079 30710 catalog_manager.cc:5484] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:49.645295 31266 raft_consensus.cc:2932] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:49.647256 31002 consensus_queue.cc:237] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:49.648661 1433 raft_consensus.cc:1261] T be25ac1c6e7242519ff636242718cb79 P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:49.648787 354 raft_consensus.cc:1261] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:49.649101 5725 consensus_queue.cc:1035] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:49.649243 4692 consensus_queue.cc:1035] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:49.650581 4692 raft_consensus.cc:2932] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:49.651002 354 raft_consensus.cc:2932] T be25ac1c6e7242519ff636242718cb79 P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:49.651376 1433 raft_consensus.cc:2932] T be25ac1c6e7242519ff636242718cb79 P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:49.652184 30700 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet be25ac1c6e7242519ff636242718cb79 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:49.652889 30713 catalog_manager.cc:5484] T be25ac1c6e7242519ff636242718cb79 P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index 8 to 9, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:49.655086 31246 tablet_service.cc:1514] Processing DeleteTablet for tablet be25ac1c6e7242519ff636242718cb79 with delete_type TABLET_DATA_TOMBSTONED (TS 14753c93125941acaad7283edc669052 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:53826
I20240502 14:00:49.655473 5732 tablet_replica.cc:330] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052: stopping tablet replica
I20240502 14:00:49.655637 5732 raft_consensus.cc:2229] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:49.655808 5732 raft_consensus.cc:2258] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:49.656400 5732 ts_tablet_manager.cc:1875] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:49.657892 5630 raft_consensus.cc:979] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f: : Instructing follower 14753c93125941acaad7283edc669052 to start an election
I20240502 14:00:49.657984 5630 raft_consensus.cc:1067] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Signalling peer 14753c93125941acaad7283edc669052 to start an election
I20240502 14:00:49.658424 5732 ts_tablet_manager.cc:1888] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 14:00:49.658424 31266 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5a3c9c93437c43b1a9c0326f2880a04e"
dest_uuid: "14753c93125941acaad7283edc669052"
from {username='slave'} at 127.24.66.65:45477
I20240502 14:00:49.658541 5732 log.cc:1196] T be25ac1c6e7242519ff636242718cb79 P 14753c93125941acaad7283edc669052: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/be25ac1c6e7242519ff636242718cb79
I20240502 14:00:49.658572 31266 raft_consensus.cc:482] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 14:00:49.658640 31266 raft_consensus.cc:3037] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:49.659040 30701 catalog_manager.cc:4830] TS 14753c93125941acaad7283edc669052 (127.24.66.68:37029): tablet be25ac1c6e7242519ff636242718cb79 (table rebalance_test_table_0 [id=a5535db3201345e78ac8467a4d987c08]) successfully deleted
I20240502 14:00:49.659406 31266 raft_consensus.cc:504] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:49.659776 31266 leader_election.cc:288] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 2 election: Requested vote from peers ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553), f769c13ece0b40179ba120dbebbaba98 (127.24.66.70:38291), 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73:34725)
I20240502 14:00:49.660152 354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3c9c93437c43b1a9c0326f2880a04e" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "f769c13ece0b40179ba120dbebbaba98"
I20240502 14:00:49.660142 1433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3c9c93437c43b1a9c0326f2880a04e" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "06d9e72a952c44b391752da7674fa6c0"
I20240502 14:00:49.660244 354 raft_consensus.cc:3037] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:49.660244 1433 raft_consensus.cc:3037] T 5a3c9c93437c43b1a9c0326f2880a04e P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:49.660523 30865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3c9c93437c43b1a9c0326f2880a04e" candidate_uuid: "14753c93125941acaad7283edc669052" candidate_term: 2 candidate_status { last_received { term: 1 index: 8 } } ignore_live_leader: true dest_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f"
I20240502 14:00:49.660646 30865 raft_consensus.cc:3032] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Stepping down as leader of term 1
I20240502 14:00:49.660677 30865 raft_consensus.cc:729] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ad5befacd63742a1b5b69fdacdb8cf8f, State: Running, Role: LEADER
I20240502 14:00:49.660790 30865 consensus_queue.cc:260] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:49.660954 30865 raft_consensus.cc:3037] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 1 FOLLOWER]: Advancing to term 2
I20240502 14:00:49.661036 354 raft_consensus.cc:2454] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 2.
I20240502 14:00:49.661036 1433 raft_consensus.cc:2454] T 5a3c9c93437c43b1a9c0326f2880a04e P 06d9e72a952c44b391752da7674fa6c0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 2.
I20240502 14:00:49.661319 31201 leader_election.cc:302] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 06d9e72a952c44b391752da7674fa6c0, 14753c93125941acaad7283edc669052, f769c13ece0b40179ba120dbebbaba98; no voters:
I20240502 14:00:49.661468 5492 raft_consensus.cc:2781] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 14:00:49.661561 5492 raft_consensus.cc:686] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [term 2 LEADER]: Becoming Leader. State: Replica: 14753c93125941acaad7283edc669052, State: Running, Role: LEADER
I20240502 14:00:49.661732 5492 consensus_queue.cc:237] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:49.661839 30865 raft_consensus.cc:2454] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14753c93125941acaad7283edc669052 in term 2.
I20240502 14:00:49.663292 30710 catalog_manager.cc:5484] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 reported cstate change: term changed from 1 to 2, leader changed from ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) to 14753c93125941acaad7283edc669052 (127.24.66.68). New cstate: current_term: 2 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:49.684909 1433 raft_consensus.cc:1203] T 8c77a03ce6a142debba75e0d387aa8ec P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20240502 14:00:49.953682 349 consensus_queue.cc:237] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:49.954744 31134 raft_consensus.cc:1261] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc [term 2 FOLLOWER]: Refusing update from remote peer f769c13ece0b40179ba120dbebbaba98: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240502 14:00:49.954744 1433 raft_consensus.cc:1261] T 1f3459ebf25f45dd810046420f5dcd81 P 06d9e72a952c44b391752da7674fa6c0 [term 2 FOLLOWER]: Refusing update from remote peer f769c13ece0b40179ba120dbebbaba98: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240502 14:00:49.955128 5718 consensus_queue.cc:1035] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:00:49.955359 5720 consensus_queue.cc:1035] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:00:49.957005 5718 raft_consensus.cc:2932] T 1f3459ebf25f45dd810046420f5dcd81 P f769c13ece0b40179ba120dbebbaba98 [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:49.957298 31134 raft_consensus.cc:2932] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:49.958431 30700 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 1f3459ebf25f45dd810046420f5dcd81 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:49.958851 1433 raft_consensus.cc:2932] T 1f3459ebf25f45dd810046420f5dcd81 P 06d9e72a952c44b391752da7674fa6c0 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:49.960112 30710 catalog_manager.cc:5484] T 1f3459ebf25f45dd810046420f5dcd81 P 8c61042437444f5f92c76109dfb229fc reported cstate change: config changed from index 8 to 10, VOTER 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66) evicted. New cstate: current_term: 2 leader_uuid: "f769c13ece0b40179ba120dbebbaba98" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "8c61042437444f5f92c76109dfb229fc" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34193 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:49.962482 30982 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f3459ebf25f45dd810046420f5dcd81 with delete_type TABLET_DATA_TOMBSTONED (TS 1fab10dcfb8542fca2b965fed6c67929 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:58604
I20240502 14:00:49.963029 5774 tablet_replica.cc:330] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929: stopping tablet replica
I20240502 14:00:49.963217 5774 raft_consensus.cc:2229] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:49.963394 5774 raft_consensus.cc:2258] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:49.964004 5774 ts_tablet_manager.cc:1875] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:49.966392 5774 ts_tablet_manager.cc:1888] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240502 14:00:49.966475 5774 log.cc:1196] T 1f3459ebf25f45dd810046420f5dcd81 P 1fab10dcfb8542fca2b965fed6c67929: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/wals/1f3459ebf25f45dd810046420f5dcd81
I20240502 14:00:49.967043 30700 catalog_manager.cc:4830] TS 1fab10dcfb8542fca2b965fed6c67929 (127.24.66.66:36395): tablet 1f3459ebf25f45dd810046420f5dcd81 (table rebalance_test_table_9 [id=5f28187b5b7a472f8429ed627f138f76]) successfully deleted
I20240502 14:00:50.170769 30865 raft_consensus.cc:1261] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 2 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 14:00:50.171128 5492 consensus_queue.cc:1035] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5befacd63742a1b5b69fdacdb8cf8f" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 39553 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:50.175958 1433 raft_consensus.cc:1261] T 5a3c9c93437c43b1a9c0326f2880a04e P 06d9e72a952c44b391752da7674fa6c0 [term 2 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 14:00:50.176290 5492 consensus_queue.cc:1035] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:50.178764 349 raft_consensus.cc:1261] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98 [term 2 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 2 index: 9. (index mismatch)
I20240502 14:00:50.179421 5492 consensus_queue.cc:1035] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:50.228660 5814 raft_consensus.cc:1050] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929: attempting to promote NON_VOTER 06d9e72a952c44b391752da7674fa6c0 to VOTER
I20240502 14:00:50.229069 5814 consensus_queue.cc:237] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:50.230327 1433 raft_consensus.cc:1261] T 8c77a03ce6a142debba75e0d387aa8ec P 06d9e72a952c44b391752da7674fa6c0 [term 1 LEARNER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:50.230533 620 raft_consensus.cc:1261] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:50.230728 31266 raft_consensus.cc:1261] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:00:50.230938 5727 consensus_queue.cc:1035] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:50.231088 5814 consensus_queue.cc:1035] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:50.231277 5816 consensus_queue.cc:1035] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:00:50.233060 5727 raft_consensus.cc:2932] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:50.233501 1433 raft_consensus.cc:2932] T 8c77a03ce6a142debba75e0d387aa8ec P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:50.235051 620 raft_consensus.cc:2932] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:50.235958 30710 catalog_manager.cc:5484] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:50.237257 31266 raft_consensus.cc:2932] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 06d9e72a952c44b391752da7674fa6c0 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } attrs { replace: true } } peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:50.239037 31002 consensus_queue.cc:237] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:50.240556 1433 raft_consensus.cc:1261] T 8c77a03ce6a142debba75e0d387aa8ec P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:50.240689 620 raft_consensus.cc:1261] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Refusing update from remote peer 1fab10dcfb8542fca2b965fed6c67929: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:00:50.241048 5814 consensus_queue.cc:1035] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:50.241187 5816 consensus_queue.cc:1035] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:00:50.242481 5727 raft_consensus.cc:2932] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:50.242977 1433 raft_consensus.cc:2932] T 8c77a03ce6a142debba75e0d387aa8ec P 06d9e72a952c44b391752da7674fa6c0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:50.244491 30700 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 8c77a03ce6a142debba75e0d387aa8ec with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:50.244515 620 raft_consensus.cc:2932] T 8c77a03ce6a142debba75e0d387aa8ec P 2bb442cde32f426bb14791852dd81802 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:50.244966 30711 catalog_manager.cc:5484] T 8c77a03ce6a142debba75e0d387aa8ec P 1fab10dcfb8542fca2b965fed6c67929 reported cstate change: config changed from index 8 to 9, VOTER 14753c93125941acaad7283edc669052 (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "1fab10dcfb8542fca2b965fed6c67929" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1fab10dcfb8542fca2b965fed6c67929" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 36395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bb442cde32f426bb14791852dd81802" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 34475 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:00:50.247120 31246 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c77a03ce6a142debba75e0d387aa8ec with delete_type TABLET_DATA_TOMBSTONED (TS 14753c93125941acaad7283edc669052 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:53826
I20240502 14:00:50.247705 5821 tablet_replica.cc:330] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052: stopping tablet replica
I20240502 14:00:50.247830 5821 raft_consensus.cc:2229] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:50.248015 5821 raft_consensus.cc:2258] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:50.248590 5821 ts_tablet_manager.cc:1875] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:50.250475 5821 ts_tablet_manager.cc:1888] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.8
I20240502 14:00:50.250543 5821 log.cc:1196] T 8c77a03ce6a142debba75e0d387aa8ec P 14753c93125941acaad7283edc669052: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/8c77a03ce6a142debba75e0d387aa8ec
I20240502 14:00:50.250996 30701 catalog_manager.cc:4830] TS 14753c93125941acaad7283edc669052 (127.24.66.68:37029): tablet 8c77a03ce6a142debba75e0d387aa8ec (table rebalance_test_table_3 [id=3fb9e83c8b7c4d90a375d14e792f650d]) successfully deleted
I20240502 14:00:50.609315 31266 consensus_queue.cc:237] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.9, Last appended by leader: 8, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } }
I20240502 14:00:50.610574 1433 raft_consensus.cc:1261] T 5a3c9c93437c43b1a9c0326f2880a04e P 06d9e72a952c44b391752da7674fa6c0 [term 2 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240502 14:00:50.610776 354 raft_consensus.cc:1261] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98 [term 2 FOLLOWER]: Refusing update from remote peer 14753c93125941acaad7283edc669052: Log matching property violated. Preceding OpId in replica: term: 2 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20240502 14:00:50.611335 5797 consensus_queue.cc:1035] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:00:50.611501 5492 consensus_queue.cc:1035] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:00:50.613260 5797 raft_consensus.cc:2932] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 [term 2 LEADER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:50.614858 30701 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5a3c9c93437c43b1a9c0326f2880a04e with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:00:50.615454 349 raft_consensus.cc:2932] T 5a3c9c93437c43b1a9c0326f2880a04e P f769c13ece0b40179ba120dbebbaba98 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:50.615747 30710 catalog_manager.cc:5484] T 5a3c9c93437c43b1a9c0326f2880a04e P 14753c93125941acaad7283edc669052 reported cstate change: config changed from index 8 to 10, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New cstate: current_term: 2 leader_uuid: "14753c93125941acaad7283edc669052" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:00:50.618213 1433 raft_consensus.cc:2932] T 5a3c9c93437c43b1a9c0326f2880a04e P 06d9e72a952c44b391752da7674fa6c0 [term 2 FOLLOWER]: Committing config change with OpId 2.10: config changed from index 8 to 10, VOTER ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "14753c93125941acaad7283edc669052" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 37029 } } peers { permanent_uuid: "f769c13ece0b40179ba120dbebbaba98" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 38291 } attrs { promote: false } } peers { permanent_uuid: "06d9e72a952c44b391752da7674fa6c0" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 34725 } attrs { promote: false } } }
I20240502 14:00:50.619609 30843 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a3c9c93437c43b1a9c0326f2880a04e with delete_type TABLET_DATA_TOMBSTONED (TS ad5befacd63742a1b5b69fdacdb8cf8f not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:45544
I20240502 14:00:50.624579 5841 tablet_replica.cc:330] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f: stopping tablet replica
I20240502 14:00:50.624760 5841 raft_consensus.cc:2229] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 14:00:50.624889 5841 raft_consensus.cc:2258] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 14:00:50.625559 5841 ts_tablet_manager.cc:1875] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:00:50.628110 5841 ts_tablet_manager.cc:1888] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.9
I20240502 14:00:50.628237 5841 log.cc:1196] T 5a3c9c93437c43b1a9c0326f2880a04e P ad5befacd63742a1b5b69fdacdb8cf8f: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/wals/5a3c9c93437c43b1a9c0326f2880a04e
I20240502 14:00:50.629030 30698 catalog_manager.cc:4830] TS ad5befacd63742a1b5b69fdacdb8cf8f (127.24.66.65:39553): tablet 5a3c9c93437c43b1a9c0326f2880a04e (table rebalance_test_table_8 [id=5cb834cd98904017a28fc7b595efc564]) successfully deleted
I20240502 14:00:50.712859 30982 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:00:50.714171 31246 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 14:00:50.725116 32671 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:50.727523 600 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:50.732892 1426 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:50.738490 449 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:50.743417 30843 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 14:00:50.746459 329 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:50.756767 31115 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 14:00:50.862598 31246 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 14:00:50.863224 1426 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:50.872546 32671 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:50.873752 30982 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:00:50.877048 449 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:50.877118 31115 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 14:00:50.877607 600 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:50.878901 329 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:50.888500 30843 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 14:00:51.131438 32671 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:51.133752 31246 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 14:00:51.133936 30982 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:00:51.144394 31115 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 14:00:51.144630 30843 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 14:00:51.153646 1426 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:51.157986 329 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:51.164763 449 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:51.173887 600 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:51.282294 1426 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:51.282675 449 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:51.286415 32671 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:51.293267 600 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:51.298017 31115 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 14:00:51.304664 30982 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:00:51.314254 31246 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 14:00:51.328044 329 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:51.328140 30843 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 14:00:51.435855 1426 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:51.438464 31246 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 14:00:51.444890 30982 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:00:51.448015 32671 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:51.449419 449 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:51.454051 31115 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 14:00:51.474349 329 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:51.485095 30843 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 14:00:51.494603 600 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:51.588148 30982 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:00:51.588754 31246 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 14:00:51.589835 32671 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:51.594173 31115 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 14:00:51.599236 449 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:51.613281 600 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:51.621127 1426 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:00:51.631872 30843 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 14:00:51.634316 329 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
2085e95de92d409c8396066e617c592b | 127.24.66.126:40727 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.24.66.84:35285 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
06d9e72a952c44b391752da7674fa6c0 | 127.24.66.73:34725 | HEALTHY | <none> | 1 | 0
14753c93125941acaad7283edc669052 | 127.24.66.68:37029 | HEALTHY | <none> | 7 | 0
1fab10dcfb8542fca2b965fed6c67929 | 127.24.66.66:36395 | HEALTHY | <none> | 5 | 0
2716cee98b1b4c4591a40e1f466520e5 | 127.24.66.69:35027 | HEALTHY | <none> | 0 | 0
2bb442cde32f426bb14791852dd81802 | 127.24.66.72:34475 | HEALTHY | <none> | 0 | 0
3daf9e0837c74a919c89e686597ff56b | 127.24.66.71:35161 | HEALTHY | <none> | 1 | 0
8c61042437444f5f92c76109dfb229fc | 127.24.66.67:34193 | HEALTHY | <none> | 8 | 0
ad5befacd63742a1b5b69fdacdb8cf8f | 127.24.66.65:39553 | HEALTHY | <none> | 7 | 0
f769c13ece0b40179ba120dbebbaba98 | 127.24.66.70:38291 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 9
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
---------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 9 server(s) checked
local_ip_for_outbound_sockets | 127.24.66.65 | experimental | 127.24.66.65:39553
local_ip_for_outbound_sockets | 127.24.66.66 | experimental | 127.24.66.66:36395
local_ip_for_outbound_sockets | 127.24.66.67 | experimental | 127.24.66.67:34193
local_ip_for_outbound_sockets | 127.24.66.68 | experimental | 127.24.66.68:37029
local_ip_for_outbound_sockets | 127.24.66.69 | experimental | 127.24.66.69:35027
local_ip_for_outbound_sockets | 127.24.66.70 | experimental | 127.24.66.70:38291
local_ip_for_outbound_sockets | 127.24.66.71 | experimental | 127.24.66.71:35161
local_ip_for_outbound_sockets | 127.24.66.72 | experimental | 127.24.66.72:34475
local_ip_for_outbound_sockets | 127.24.66.73 | experimental | 127.24.66.73:34725
never_fsync | true | unsafe,advanced | all 9 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 9 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 9 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 9 server(s) checked
server_dump_info_format | pb | hidden | all 9 server(s) checked
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.24.66.65:39553
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.24.66.66:36395
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.24.66.67:34193
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.24.66.68:37029
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.24.66.69:35027
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.24.66.70:38291
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.24.66.71:35161
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data/info.pb | hidden | 127.24.66.72:34475
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.TserverAddedDuringRebalancingTest.TserverStarts_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/data/info.pb | hidden | 127.24.66.73:34725
tablet_copy_download_file_inject_latency_ms | 1500 | hidden | all 9 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.24.66.84:35285 | all 9 server(s) checked
time_source | builtin | all 9 server(s) checked
Version Summary
Version | Servers
-----------------+--------------------------
1.18.0-SNAPSHOT | all 10 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
rebalance_test_table_0 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_5 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_6 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_7 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_8 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_9 | 3 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 10
First Quartile | 10
Median | 10
Third Quartile | 10
Maximum | 10
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 9
Tables | 10
Tablets | 30
Replicas | 90
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240502 14:00:51.664918 24841 log_verifier.cc:126] Checking tablet 03fb6c61ce4c4ccdafdc9ca7a40db4f5
I20240502 14:00:51.690217 24841 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 03fb6c61ce4c4ccdafdc9ca7a40db4f5
I20240502 14:00:51.690274 24841 log_verifier.cc:126] Checking tablet 08588e44069d4044b2d09eb02d252f82
I20240502 14:00:51.715011 24841 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 08588e44069d4044b2d09eb02d252f82
I20240502 14:00:51.715071 24841 log_verifier.cc:126] Checking tablet 1f3459ebf25f45dd810046420f5dcd81
I20240502 14:00:51.740329 24841 log_verifier.cc:177] Verified matching terms for 10 ops in tablet 1f3459ebf25f45dd810046420f5dcd81
I20240502 14:00:51.740444 24841 log_verifier.cc:126] Checking tablet 2051076a81274b688554c218c107ee1c
I20240502 14:00:51.764920 24841 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 2051076a81274b688554c218c107ee1c
I20240502 14:00:51.764981 24841 log_verifier.cc:126] Checking tablet 2e02dc2a98d04bddafdd1b933d89b4dd
I20240502 14:00:51.790927 24841 log_verifier.cc:177] Verified matching terms for 10 ops in tablet 2e02dc2a98d04bddafdd1b933d89b4dd
I20240502 14:00:51.791004 24841 log_verifier.cc:126] Checking tablet 2e07f0adca594453bad9433ed0017580
I20240502 14:00:51.817116 24841 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 2e07f0adca594453bad9433ed0017580
I20240502 14:00:51.817178 24841 log_verifier.cc:126] Checking tablet 342cec2187904922890444a7e5dd181b
I20240502 14:00:51.845710 24841 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 342cec2187904922890444a7e5dd181b
I20240502 14:00:51.845770 24841 log_verifier.cc:126] Checking tablet 39239edeafb84b5caaa5dceea06fd4b0
I20240502 14:00:51.870533 24841 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 39239edeafb84b5caaa5dceea06fd4b0
I20240502 14:00:51.870590 24841 log_verifier.cc:126] Checking tablet 42ecb8fd71404d7b9eb25af03d2200af
I20240502 14:00:51.894981 24841 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 42ecb8fd71404d7b9eb25af03d2200af
I20240502 14:00:51.895040 24841 log_verifier.cc:126] Checking tablet 46b0a06884ad48da9b060d2810d5c782
I20240502 14:00:51.924649 24841 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 46b0a06884ad48da9b060d2810d5c782
I20240502 14:00:51.924728 24841 log_verifier.cc:126] Checking tablet 4a12b8eaab7943c3bc9ee48c2a2307d0
I20240502 14:00:51.961655 24841 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 4a12b8eaab7943c3bc9ee48c2a2307d0
I20240502 14:00:51.961735 24841 log_verifier.cc:126] Checking tablet 5702766b82f24e96a4cb5da5f051fc3f
I20240502 14:00:51.998597 24841 log_verifier.cc:177] Verified matching terms for 7 ops in tablet 5702766b82f24e96a4cb5da5f051fc3f
I20240502 14:00:51.998679 24841 log_verifier.cc:126] Checking tablet 5a3c9c93437c43b1a9c0326f2880a04e
I20240502 14:00:52.036686 24841 log_verifier.cc:177] Verified matching terms for 10 ops in tablet 5a3c9c93437c43b1a9c0326f2880a04e
I20240502 14:00:52.036772 24841 log_verifier.cc:126] Checking tablet 66e2378d5c0f400299a170df8a939577
I20240502 14:00:52.074457 24841 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 66e2378d5c0f400299a170df8a939577
I20240502 14:00:52.074539 24841 log_verifier.cc:126] Checking tablet 8c77a03ce6a142debba75e0d387aa8ec
I20240502 14:00:52.101024 24841 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 8c77a03ce6a142debba75e0d387aa8ec
I20240502 14:00:52.101084 24841 log_verifier.cc:126] Checking tablet 8da28cc4aef043baaa2282f84afd4abf
I20240502 14:00:52.127202 24841 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 8da28cc4aef043baaa2282f84afd4abf
I20240502 14:00:52.127281 24841 log_verifier.cc:126] Checking tablet 8ec88a191b8d4188985b5e3c91a2a031
I20240502 14:00:52.158187 24841 log_verifier.cc:177] Verified matching terms for 10 ops in tablet 8ec88a191b8d4188985b5e3c91a2a031
I20240502 14:00:52.158248 24841 log_verifier.cc:126] Checking tablet 903f5d6c1faf4267b7f7d7dc32670a52
I20240502 14:00:52.181788 24841 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 903f5d6c1faf4267b7f7d7dc32670a52
I20240502 14:00:52.181850 24841 log_verifier.cc:126] Checking tablet 954fa01ea4d04842b0218b33de347aa1
I20240502 14:00:52.207244 24841 log_verifier.cc:177] Verified matching terms for 9 ops in tablet 954fa01ea4d04842b0218b33de347aa1
I20240502 14:00:52.207317 24841 log_verifier.cc:126] Checking tablet aaca0594b68146cdbd18946f123693be
I20240502 14:00:52.232228 24841 log_verifier.cc:177] Verified matching terms for 9 ops in tablet aaca0594b68146cdbd18946f123693be
I20240502 14:00:52.232285 24841 log_verifier.cc:126] Checking tablet b5b3cf5fbe3d4d999cbf910c8e0ca62e
I20240502 14:00:52.256793 24841 log_verifier.cc:177] Verified matching terms for 6 ops in tablet b5b3cf5fbe3d4d999cbf910c8e0ca62e
I20240502 14:00:52.256855 24841 log_verifier.cc:126] Checking tablet be25ac1c6e7242519ff636242718cb79
I20240502 14:00:52.282135 24841 log_verifier.cc:177] Verified matching terms for 9 ops in tablet be25ac1c6e7242519ff636242718cb79
I20240502 14:00:52.282197 24841 log_verifier.cc:126] Checking tablet c0b9178646764824b17cbd842fbb0ff2
I20240502 14:00:52.305677 24841 log_verifier.cc:177] Verified matching terms for 6 ops in tablet c0b9178646764824b17cbd842fbb0ff2
I20240502 14:00:52.305752 24841 log_verifier.cc:126] Checking tablet c43ff8fc0b4f4f799a6f1bbd677166da
I20240502 14:00:52.332219 24841 log_verifier.cc:177] Verified matching terms for 6 ops in tablet c43ff8fc0b4f4f799a6f1bbd677166da
I20240502 14:00:52.332284 24841 log_verifier.cc:126] Checking tablet cc64ea7db2064251bed364b800b36b87
I20240502 14:00:52.357425 24841 log_verifier.cc:177] Verified matching terms for 9 ops in tablet cc64ea7db2064251bed364b800b36b87
I20240502 14:00:52.357506 24841 log_verifier.cc:126] Checking tablet d0409950632d4eeead8a826034b5a9e4
I20240502 14:00:52.382339 24841 log_verifier.cc:177] Verified matching terms for 10 ops in tablet d0409950632d4eeead8a826034b5a9e4
I20240502 14:00:52.382401 24841 log_verifier.cc:126] Checking tablet da2f3b3481da476d98c97bf35f055c2e
I20240502 14:00:52.407229 24841 log_verifier.cc:177] Verified matching terms for 10 ops in tablet da2f3b3481da476d98c97bf35f055c2e
I20240502 14:00:52.407310 24841 log_verifier.cc:126] Checking tablet e109ef8f5d764df7969642dc6de361a2
I20240502 14:00:52.431862 24841 log_verifier.cc:177] Verified matching terms for 6 ops in tablet e109ef8f5d764df7969642dc6de361a2
I20240502 14:00:52.431941 24841 log_verifier.cc:126] Checking tablet efd8e3b804cf41bebf5e629b65571979
I20240502 14:00:52.457530 24841 log_verifier.cc:177] Verified matching terms for 9 ops in tablet efd8e3b804cf41bebf5e629b65571979
I20240502 14:00:52.457659 24841 log_verifier.cc:126] Checking tablet f571119d6f624fe88bf1effeaf466085
I20240502 14:00:52.482348 24841 log_verifier.cc:177] Verified matching terms for 9 ops in tablet f571119d6f624fe88bf1effeaf466085
I20240502 14:00:52.482801 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 30779
I20240502 14:00:52.490872 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 30917
I20240502 14:00:52.497967 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 31052
I20240502 14:00:52.505045 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 31184
I20240502 14:00:52.511891 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 32609
I20240502 14:00:52.517810 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 32740
I20240502 14:00:52.523337 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 405
I20240502 14:00:52.528905 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 537
I20240502 14:00:52.534466 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 1311
I20240502 14:00:52.540057 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 30682
2024-05-02T14:00:52Z chronyd exiting
[ OK ] TserverAddedDuringRebalancingTest.TserverStarts/1 (29992 ms)
[----------] 1 test from TserverAddedDuringRebalancingTest (29992 ms total)
[----------] 1 test from RebalancerAndSingleReplicaTablets
[ RUN ] RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove/5
I20240502 14:00:52.645025 24841 test_util.cc:268] Using random seed: 133899420
I20240502 14:00:52.645727 24841 ts_itest-base.cc:112] Starting cluster with:
I20240502 14:00:52.645778 24841 ts_itest-base.cc:113] --------------
I20240502 14:00:52.645797 24841 ts_itest-base.cc:114] 4 tablet servers
I20240502 14:00:52.645813 24841 ts_itest-base.cc:115] 1 replicas per TS
I20240502 14:00:52.645830 24841 ts_itest-base.cc:116] --------------
2024-05-02T14:00:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T14:00:52Z Disabled control of system clock
I20240502 14:00:52.653359 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:34755
--webserver_interface=127.24.66.126
--webserver_port=0
--builtin_ntp_servers=127.24.66.84:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:34755
--raft_prepare_replacement_before_eviction=true
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false with env {}
W20240502 14:00:52.749596 6113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:52.749773 6113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:52.749825 6113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:52.752525 6113 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:52.752573 6113 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 14:00:52.752593 6113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:52.752609 6113 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 14:00:52.752628 6113 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 14:00:52.755869 6113 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:34755
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:34755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.6113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:52.756500 6113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:52.757417 6113 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:52.764299 6119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:52.764866 6118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:52.765558 6113 server_base.cc:1034] running on GCE node
W20240502 14:00:52.766117 6121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:52.766494 6113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:52.767227 6113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:52.768383 6113 hybrid_clock.cc:648] HybridClock initialized: now 1714658452768369 us; error 34 us; skew 500 ppm
I20240502 14:00:52.769860 6113 webserver.cc:460] Webserver started at http://127.24.66.126:35605/ using document root <none> and password file <none>
I20240502 14:00:52.770308 6113 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:52.770373 6113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:52.770542 6113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:52.771936 6113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "9286b3b7f4204edabe8203e0862dd75a"
format_stamp: "Formatted at 2024-05-02 14:00:52 on dist-test-slave-5npg"
I20240502 14:00:52.772311 6113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "9286b3b7f4204edabe8203e0862dd75a"
format_stamp: "Formatted at 2024-05-02 14:00:52 on dist-test-slave-5npg"
I20240502 14:00:52.774943 6113 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:52.776777 6127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:52.777652 6113 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:52.777752 6113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
uuid: "9286b3b7f4204edabe8203e0862dd75a"
format_stamp: "Formatted at 2024-05-02 14:00:52 on dist-test-slave-5npg"
I20240502 14:00:52.777824 6113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:52.805251 6113 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:52.805691 6113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:52.805866 6113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:52.811450 6113 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:34755
I20240502 14:00:52.811460 6179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:34755 every 8 connection(s)
I20240502 14:00:52.812219 6113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 14:00:52.814261 6180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:52.818486 6180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9286b3b7f4204edabe8203e0862dd75a: Bootstrap starting.
I20240502 14:00:52.818677 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 6113
I20240502 14:00:52.818784 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 14:00:52.820495 6180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9286b3b7f4204edabe8203e0862dd75a: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:52.821043 6180 log.cc:824] T 00000000000000000000000000000000 P 9286b3b7f4204edabe8203e0862dd75a: Log is configured to *not* fsync() on all Append() calls
I20240502 14:00:52.822314 6180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9286b3b7f4204edabe8203e0862dd75a: No bootstrap required, opened a new log
I20240502 14:00:52.824489 6180 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9286b3b7f4204edabe8203e0862dd75a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9286b3b7f4204edabe8203e0862dd75a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34755 } }
I20240502 14:00:52.824640 6180 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9286b3b7f4204edabe8203e0862dd75a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:52.824707 6180 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9286b3b7f4204edabe8203e0862dd75a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9286b3b7f4204edabe8203e0862dd75a, State: Initialized, Role: FOLLOWER
I20240502 14:00:52.825098 6180 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9286b3b7f4204edabe8203e0862dd75a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9286b3b7f4204edabe8203e0862dd75a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34755 } }
I20240502 14:00:52.825229 6180 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9286b3b7f4204edabe8203e0862dd75a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 14:00:52.825289 6180 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9286b3b7f4204edabe8203e0862dd75a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 14:00:52.825381 6180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9286b3b7f4204edabe8203e0862dd75a [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:52.826023 6180 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9286b3b7f4204edabe8203e0862dd75a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9286b3b7f4204edabe8203e0862dd75a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34755 } }
I20240502 14:00:52.826330 6180 leader_election.cc:302] T 00000000000000000000000000000000 P 9286b3b7f4204edabe8203e0862dd75a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9286b3b7f4204edabe8203e0862dd75a; no voters:
I20240502 14:00:52.826573 6180 leader_election.cc:288] T 00000000000000000000000000000000 P 9286b3b7f4204edabe8203e0862dd75a [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 14:00:52.826896 6185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9286b3b7f4204edabe8203e0862dd75a [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:52.827087 6185 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9286b3b7f4204edabe8203e0862dd75a [term 1 LEADER]: Becoming Leader. State: Replica: 9286b3b7f4204edabe8203e0862dd75a, State: Running, Role: LEADER
I20240502 14:00:52.827316 6185 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9286b3b7f4204edabe8203e0862dd75a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9286b3b7f4204edabe8203e0862dd75a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34755 } }
I20240502 14:00:52.827440 6180 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9286b3b7f4204edabe8203e0862dd75a [sys.catalog]: configured and running, proceeding with master startup.
I20240502 14:00:52.828634 6186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9286b3b7f4204edabe8203e0862dd75a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9286b3b7f4204edabe8203e0862dd75a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9286b3b7f4204edabe8203e0862dd75a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34755 } } }
I20240502 14:00:52.828755 6186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9286b3b7f4204edabe8203e0862dd75a [sys.catalog]: This master's current role is: LEADER
I20240502 14:00:52.829052 6187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9286b3b7f4204edabe8203e0862dd75a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9286b3b7f4204edabe8203e0862dd75a. Latest consensus state: current_term: 1 leader_uuid: "9286b3b7f4204edabe8203e0862dd75a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9286b3b7f4204edabe8203e0862dd75a" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34755 } } }
I20240502 14:00:52.829592 6187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9286b3b7f4204edabe8203e0862dd75a [sys.catalog]: This master's current role is: LEADER
W20240502 14:00:52.831286 6200 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9286b3b7f4204edabe8203e0862dd75a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240502 14:00:52.831360 6200 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20240502 14:00:52.831459 6195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 14:00:52.832096 6195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 14:00:52.836174 6195 catalog_manager.cc:1347] Generated new cluster ID: 01c7fc2259b84f25b5b6087cf31eb6b6
I20240502 14:00:52.836232 6195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 14:00:52.848595 6195 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 14:00:52.849284 6195 catalog_manager.cc:1504] Loading token signing keys...
I20240502 14:00:52.854008 6195 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 9286b3b7f4204edabe8203e0862dd75a: Generated new TSK 0
I20240502 14:00:52.854574 6195 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 14:00:52.865970 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:34755
--builtin_ntp_servers=127.24.66.84:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 14:00:52.977102 6204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:52.977279 6204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:52.977339 6204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:52.980114 6204 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:52.980165 6204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:52.980249 6204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20240502 14:00:52.983477 6204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:34755
--never_fsync=true
--heap_profile_path=/tmp/kudu.6204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:52.984282 6204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:52.985373 6204 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:52.993471 6212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:52.994093 6209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:52.994755 6210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:52.995432 6204 server_base.cc:1034] running on GCE node
I20240502 14:00:52.995750 6204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:52.996232 6204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:52.997404 6204 hybrid_clock.cc:648] HybridClock initialized: now 1714658452997385 us; error 34 us; skew 500 ppm
I20240502 14:00:52.999004 6204 webserver.cc:460] Webserver started at http://127.24.66.65:40117/ using document root <none> and password file <none>
I20240502 14:00:52.999485 6204 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:52.999536 6204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:52.999688 6204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:53.001235 6204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "e3c8043a08994c0bbf451c44633e2860"
format_stamp: "Formatted at 2024-05-02 14:00:52 on dist-test-slave-5npg"
I20240502 14:00:53.001673 6204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "e3c8043a08994c0bbf451c44633e2860"
format_stamp: "Formatted at 2024-05-02 14:00:52 on dist-test-slave-5npg"
I20240502 14:00:53.004586 6204 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 14:00:53.006433 6218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:53.007278 6204 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:53.007385 6204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "e3c8043a08994c0bbf451c44633e2860"
format_stamp: "Formatted at 2024-05-02 14:00:52 on dist-test-slave-5npg"
I20240502 14:00:53.007459 6204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:53.028765 6204 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:53.029219 6204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:53.029387 6204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:53.029860 6204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:53.030591 6204 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:53.030647 6204 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:53.030704 6204 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:53.030730 6204 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:53.038564 6204 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:42853
I20240502 14:00:53.038591 6331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:42853 every 8 connection(s)
I20240502 14:00:53.039340 6204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 14:00:53.041911 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 6204
I20240502 14:00:53.042032 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 14:00:53.044085 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:34755
--builtin_ntp_servers=127.24.66.84:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:00:53.048218 6332 heartbeater.cc:347] Connected to a master server at 127.24.66.126:34755
I20240502 14:00:53.048476 6332 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:53.049082 6332 heartbeater.cc:511] Master 127.24.66.126:34755 requested a full tablet report, sending...
I20240502 14:00:53.050628 6144 ts_manager.cc:194] Registered new tserver with Master: e3c8043a08994c0bbf451c44633e2860 (127.24.66.65:42853)
I20240502 14:00:53.052276 6144 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.65:53309
W20240502 14:00:53.143879 6335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:53.144033 6335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:53.144069 6335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:53.146785 6335 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:53.146836 6335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:53.146898 6335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20240502 14:00:53.150122 6335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:34755
--never_fsync=true
--heap_profile_path=/tmp/kudu.6335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:53.150749 6335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:53.151631 6335 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:53.159432 6340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:53.160002 6343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:53.160619 6341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:53.161326 6335 server_base.cc:1034] running on GCE node
I20240502 14:00:53.161667 6335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:53.162113 6335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:53.163260 6335 hybrid_clock.cc:648] HybridClock initialized: now 1714658453163252 us; error 25 us; skew 500 ppm
I20240502 14:00:53.164832 6335 webserver.cc:460] Webserver started at http://127.24.66.66:38649/ using document root <none> and password file <none>
I20240502 14:00:53.165294 6335 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:53.165344 6335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:53.165510 6335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:53.166961 6335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d"
format_stamp: "Formatted at 2024-05-02 14:00:53 on dist-test-slave-5npg"
I20240502 14:00:53.167385 6335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d"
format_stamp: "Formatted at 2024-05-02 14:00:53 on dist-test-slave-5npg"
I20240502 14:00:53.170241 6335 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 14:00:53.172049 6349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:53.172930 6335 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:53.173013 6335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d"
format_stamp: "Formatted at 2024-05-02 14:00:53 on dist-test-slave-5npg"
I20240502 14:00:53.173094 6335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:53.198774 6335 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:53.199261 6335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:53.199402 6335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:53.199852 6335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:53.200685 6335 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:53.200729 6335 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:53.200770 6335 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:53.200803 6335 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:53.208546 6335 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:42259
I20240502 14:00:53.208576 6462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:42259 every 8 connection(s)
I20240502 14:00:53.209355 6335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 14:00:53.210181 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 6335
I20240502 14:00:53.210294 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 14:00:53.212632 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:34755
--builtin_ntp_servers=127.24.66.84:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:00:53.218504 6463 heartbeater.cc:347] Connected to a master server at 127.24.66.126:34755
I20240502 14:00:53.218727 6463 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:53.219339 6463 heartbeater.cc:511] Master 127.24.66.126:34755 requested a full tablet report, sending...
I20240502 14:00:53.220321 6144 ts_manager.cc:194] Registered new tserver with Master: 9457d6a3e2e14ca0a3f3ed1e0467480d (127.24.66.66:42259)
I20240502 14:00:53.220865 6144 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.66:49503
W20240502 14:00:53.312695 6466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:53.312860 6466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:53.312909 6466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:53.315594 6466 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:53.315642 6466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:53.315699 6466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20240502 14:00:53.319041 6466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:34755
--never_fsync=true
--heap_profile_path=/tmp/kudu.6466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:53.319721 6466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:53.320659 6466 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:53.328135 6472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:53.328737 6471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:53.329314 6474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:53.330050 6466 server_base.cc:1034] running on GCE node
I20240502 14:00:53.330318 6466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:53.330763 6466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:53.331899 6466 hybrid_clock.cc:648] HybridClock initialized: now 1714658453331879 us; error 33 us; skew 500 ppm
I20240502 14:00:53.333390 6466 webserver.cc:460] Webserver started at http://127.24.66.67:36963/ using document root <none> and password file <none>
I20240502 14:00:53.333832 6466 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:53.333899 6466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:53.334053 6466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:53.335419 6466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "3a41f2428d9643b29cd86aff8d3faac7"
format_stamp: "Formatted at 2024-05-02 14:00:53 on dist-test-slave-5npg"
I20240502 14:00:53.335819 6466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "3a41f2428d9643b29cd86aff8d3faac7"
format_stamp: "Formatted at 2024-05-02 14:00:53 on dist-test-slave-5npg"
I20240502 14:00:53.338750 6466 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20240502 14:00:53.340600 6480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:53.341408 6466 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240502 14:00:53.341495 6466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "3a41f2428d9643b29cd86aff8d3faac7"
format_stamp: "Formatted at 2024-05-02 14:00:53 on dist-test-slave-5npg"
I20240502 14:00:53.341564 6466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:53.349798 6466 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:53.350224 6466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:53.350361 6466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:53.350771 6466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:53.351475 6466 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:53.351521 6466 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:53.351570 6466 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:53.351603 6466 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:53.359234 6466 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:41947
I20240502 14:00:53.359254 6593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:41947 every 8 connection(s)
I20240502 14:00:53.359949 6466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 14:00:53.367125 6594 heartbeater.cc:347] Connected to a master server at 127.24.66.126:34755
I20240502 14:00:53.367313 6594 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:53.367722 6594 heartbeater.cc:511] Master 127.24.66.126:34755 requested a full tablet report, sending...
I20240502 14:00:53.368080 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 6466
I20240502 14:00:53.368185 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 14:00:53.368598 6144 ts_manager.cc:194] Registered new tserver with Master: 3a41f2428d9643b29cd86aff8d3faac7 (127.24.66.67:41947)
I20240502 14:00:53.369161 6144 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.67:58677
I20240502 14:00:53.370146 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.68:0
--local_ip_for_outbound_sockets=127.24.66.68
--webserver_interface=127.24.66.68
--webserver_port=0
--tserver_master_addrs=127.24.66.126:34755
--builtin_ntp_servers=127.24.66.84:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 14:00:53.463855 6597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:53.464008 6597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:53.464046 6597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:53.466897 6597 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:53.466949 6597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:53.467023 6597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.68
I20240502 14:00:53.470366 6597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.24.66.68
--webserver_port=0
--tserver_master_addrs=127.24.66.126:34755
--never_fsync=true
--heap_profile_path=/tmp/kudu.6597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.68
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:53.471017 6597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:53.471909 6597 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:53.479532 6602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:53.480121 6603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:53.480652 6605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:53.481317 6597 server_base.cc:1034] running on GCE node
I20240502 14:00:53.481643 6597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:53.482123 6597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:53.483258 6597 hybrid_clock.cc:648] HybridClock initialized: now 1714658453483248 us; error 34 us; skew 500 ppm
I20240502 14:00:53.484730 6597 webserver.cc:460] Webserver started at http://127.24.66.68:43495/ using document root <none> and password file <none>
I20240502 14:00:53.485168 6597 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:53.485224 6597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:53.485376 6597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:53.486765 6597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "18f67436b19d4f96aeac62e21dbd0d9e"
format_stamp: "Formatted at 2024-05-02 14:00:53 on dist-test-slave-5npg"
I20240502 14:00:53.487181 6597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "18f67436b19d4f96aeac62e21dbd0d9e"
format_stamp: "Formatted at 2024-05-02 14:00:53 on dist-test-slave-5npg"
I20240502 14:00:53.489931 6597 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 14:00:53.491668 6611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:53.492528 6597 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:53.492628 6597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "18f67436b19d4f96aeac62e21dbd0d9e"
format_stamp: "Formatted at 2024-05-02 14:00:53 on dist-test-slave-5npg"
I20240502 14:00:53.492699 6597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:53.504614 6597 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:53.505002 6597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:53.505127 6597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:53.505518 6597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:53.506224 6597 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:53.506268 6597 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:53.506322 6597 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:53.506356 6597 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:53.513865 6597 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.68:42179
I20240502 14:00:53.513911 6725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.68:42179 every 8 connection(s)
I20240502 14:00:53.514590 6597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 14:00:53.514925 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 6597
I20240502 14:00:53.515053 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 14:00:53.521857 6726 heartbeater.cc:347] Connected to a master server at 127.24.66.126:34755
I20240502 14:00:53.522013 6726 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:53.522392 6726 heartbeater.cc:511] Master 127.24.66.126:34755 requested a full tablet report, sending...
I20240502 14:00:53.523155 6143 ts_manager.cc:194] Registered new tserver with Master: 18f67436b19d4f96aeac62e21dbd0d9e (127.24.66.68:42179)
I20240502 14:00:53.523662 6143 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.68:43155
I20240502 14:00:53.528769 24841 external_mini_cluster.cc:930] 4 TS(s) registered with all masters
I20240502 14:00:53.536012 6143 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45668:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240502 14:00:53.549501 6266 tablet_service.cc:1467] Processing CreateTablet for tablet ee6a8e1b6e17476fb5cccf3bf28096cd (DEFAULT_TABLE table=TestTable [id=11cf0456c1dc4749b0fe41d093346916]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:53.550227 6266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee6a8e1b6e17476fb5cccf3bf28096cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:53.554677 6739 tablet_bootstrap.cc:492] T ee6a8e1b6e17476fb5cccf3bf28096cd P e3c8043a08994c0bbf451c44633e2860: Bootstrap starting.
I20240502 14:00:53.556536 6739 tablet_bootstrap.cc:654] T ee6a8e1b6e17476fb5cccf3bf28096cd P e3c8043a08994c0bbf451c44633e2860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:53.557123 6739 log.cc:824] T ee6a8e1b6e17476fb5cccf3bf28096cd P e3c8043a08994c0bbf451c44633e2860: Log is configured to *not* fsync() on all Append() calls
I20240502 14:00:53.558461 6739 tablet_bootstrap.cc:492] T ee6a8e1b6e17476fb5cccf3bf28096cd P e3c8043a08994c0bbf451c44633e2860: No bootstrap required, opened a new log
I20240502 14:00:53.558619 6739 ts_tablet_manager.cc:1367] T ee6a8e1b6e17476fb5cccf3bf28096cd P e3c8043a08994c0bbf451c44633e2860: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 14:00:53.561328 6739 raft_consensus.cc:348] T ee6a8e1b6e17476fb5cccf3bf28096cd P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:53.561483 6739 raft_consensus.cc:374] T ee6a8e1b6e17476fb5cccf3bf28096cd P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:53.561545 6739 raft_consensus.cc:729] T ee6a8e1b6e17476fb5cccf3bf28096cd P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3c8043a08994c0bbf451c44633e2860, State: Initialized, Role: FOLLOWER
I20240502 14:00:53.561887 6739 consensus_queue.cc:260] T ee6a8e1b6e17476fb5cccf3bf28096cd P e3c8043a08994c0bbf451c44633e2860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:53.562023 6739 raft_consensus.cc:388] T ee6a8e1b6e17476fb5cccf3bf28096cd P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 14:00:53.562079 6739 raft_consensus.cc:482] T ee6a8e1b6e17476fb5cccf3bf28096cd P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 14:00:53.562150 6739 raft_consensus.cc:3037] T ee6a8e1b6e17476fb5cccf3bf28096cd P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:53.562767 6739 raft_consensus.cc:504] T ee6a8e1b6e17476fb5cccf3bf28096cd P e3c8043a08994c0bbf451c44633e2860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:53.563009 6739 leader_election.cc:302] T ee6a8e1b6e17476fb5cccf3bf28096cd P e3c8043a08994c0bbf451c44633e2860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3c8043a08994c0bbf451c44633e2860; no voters:
I20240502 14:00:53.563217 6739 leader_election.cc:288] T ee6a8e1b6e17476fb5cccf3bf28096cd P e3c8043a08994c0bbf451c44633e2860 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 14:00:53.563324 6741 raft_consensus.cc:2781] T ee6a8e1b6e17476fb5cccf3bf28096cd P e3c8043a08994c0bbf451c44633e2860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:53.563510 6741 raft_consensus.cc:686] T ee6a8e1b6e17476fb5cccf3bf28096cd P e3c8043a08994c0bbf451c44633e2860 [term 1 LEADER]: Becoming Leader. State: Replica: e3c8043a08994c0bbf451c44633e2860, State: Running, Role: LEADER
I20240502 14:00:53.563748 6332 heartbeater.cc:503] Master 127.24.66.126:34755 was elected leader, sending a full tablet report...
I20240502 14:00:53.563783 6741 consensus_queue.cc:237] T ee6a8e1b6e17476fb5cccf3bf28096cd P e3c8043a08994c0bbf451c44633e2860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:53.564277 6739 ts_tablet_manager.cc:1398] T ee6a8e1b6e17476fb5cccf3bf28096cd P e3c8043a08994c0bbf451c44633e2860: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20240502 14:00:53.566598 6143 catalog_manager.cc:5484] T ee6a8e1b6e17476fb5cccf3bf28096cd P e3c8043a08994c0bbf451c44633e2860 reported cstate change: term changed from 0 to 1, leader changed from <none> to e3c8043a08994c0bbf451c44633e2860 (127.24.66.65). New cstate: current_term: 1 leader_uuid: "e3c8043a08994c0bbf451c44633e2860" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:53.575234 24841 external_mini_cluster.cc:930] 4 TS(s) registered with all masters
I20240502 14:00:53.576388 24841 ts_itest-base.cc:233] Waiting for 1 tablets on tserver e3c8043a08994c0bbf451c44633e2860 to finish bootstrapping
I20240502 14:00:53.581056 24841 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 9457d6a3e2e14ca0a3f3ed1e0467480d to finish bootstrapping
I20240502 14:00:53.586009 24841 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 3a41f2428d9643b29cd86aff8d3faac7 to finish bootstrapping
I20240502 14:00:53.590752 24841 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 18f67436b19d4f96aeac62e21dbd0d9e to finish bootstrapping
I20240502 14:00:53.595525 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 6466
I20240502 14:00:53.601336 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 6597
I20240502 14:00:54.223001 6463 heartbeater.cc:503] Master 127.24.66.126:34755 was elected leader, sending a full tablet report...
I20240502 14:00:57.357322 6143 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45668:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:57.364140 6266 tablet_service.cc:1467] Processing CreateTablet for tablet 069c5651bc4c4122942fcbb946f58044 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5576c7e6675f4db7966efa79c352fd53]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:57.364503 6266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 069c5651bc4c4122942fcbb946f58044. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:57.366623 6756 tablet_bootstrap.cc:492] T 069c5651bc4c4122942fcbb946f58044 P e3c8043a08994c0bbf451c44633e2860: Bootstrap starting.
I20240502 14:00:57.367527 6756 tablet_bootstrap.cc:654] T 069c5651bc4c4122942fcbb946f58044 P e3c8043a08994c0bbf451c44633e2860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:57.369086 6756 tablet_bootstrap.cc:492] T 069c5651bc4c4122942fcbb946f58044 P e3c8043a08994c0bbf451c44633e2860: No bootstrap required, opened a new log
I20240502 14:00:57.369230 6756 ts_tablet_manager.cc:1367] T 069c5651bc4c4122942fcbb946f58044 P e3c8043a08994c0bbf451c44633e2860: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 14:00:57.368983 6397 tablet_service.cc:1467] Processing CreateTablet for tablet 967d1f826a29473184273171be78fdfc (DEFAULT_TABLE table=rebalance_test_table_0 [id=5576c7e6675f4db7966efa79c352fd53]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:57.368970 6396 tablet_service.cc:1467] Processing CreateTablet for tablet d5a0c9b9d46c469db421cce9131435df (DEFAULT_TABLE table=rebalance_test_table_0 [id=5576c7e6675f4db7966efa79c352fd53]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:57.369741 6756 raft_consensus.cc:348] T 069c5651bc4c4122942fcbb946f58044 P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:57.369824 6756 raft_consensus.cc:374] T 069c5651bc4c4122942fcbb946f58044 P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:57.369855 6756 raft_consensus.cc:729] T 069c5651bc4c4122942fcbb946f58044 P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3c8043a08994c0bbf451c44633e2860, State: Initialized, Role: FOLLOWER
I20240502 14:00:57.369992 6396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5a0c9b9d46c469db421cce9131435df. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:57.369989 6756 consensus_queue.cc:260] T 069c5651bc4c4122942fcbb946f58044 P e3c8043a08994c0bbf451c44633e2860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:57.370070 6756 raft_consensus.cc:388] T 069c5651bc4c4122942fcbb946f58044 P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 14:00:57.370111 6756 raft_consensus.cc:482] T 069c5651bc4c4122942fcbb946f58044 P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 14:00:57.370162 6756 raft_consensus.cc:3037] T 069c5651bc4c4122942fcbb946f58044 P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:57.370327 6397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 967d1f826a29473184273171be78fdfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:57.370777 6756 raft_consensus.cc:504] T 069c5651bc4c4122942fcbb946f58044 P e3c8043a08994c0bbf451c44633e2860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:57.370867 6756 leader_election.cc:302] T 069c5651bc4c4122942fcbb946f58044 P e3c8043a08994c0bbf451c44633e2860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3c8043a08994c0bbf451c44633e2860; no voters:
I20240502 14:00:57.370997 6756 leader_election.cc:288] T 069c5651bc4c4122942fcbb946f58044 P e3c8043a08994c0bbf451c44633e2860 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 14:00:57.371225 6759 raft_consensus.cc:2781] T 069c5651bc4c4122942fcbb946f58044 P e3c8043a08994c0bbf451c44633e2860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:57.371304 6756 ts_tablet_manager.cc:1398] T 069c5651bc4c4122942fcbb946f58044 P e3c8043a08994c0bbf451c44633e2860: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240502 14:00:57.371536 6759 raft_consensus.cc:686] T 069c5651bc4c4122942fcbb946f58044 P e3c8043a08994c0bbf451c44633e2860 [term 1 LEADER]: Becoming Leader. State: Replica: e3c8043a08994c0bbf451c44633e2860, State: Running, Role: LEADER
I20240502 14:00:57.371719 6759 consensus_queue.cc:237] T 069c5651bc4c4122942fcbb946f58044 P e3c8043a08994c0bbf451c44633e2860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:57.373006 6143 catalog_manager.cc:5484] T 069c5651bc4c4122942fcbb946f58044 P e3c8043a08994c0bbf451c44633e2860 reported cstate change: term changed from 0 to 1, leader changed from <none> to e3c8043a08994c0bbf451c44633e2860 (127.24.66.65). New cstate: current_term: 1 leader_uuid: "e3c8043a08994c0bbf451c44633e2860" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:57.375684 6763 tablet_bootstrap.cc:492] T d5a0c9b9d46c469db421cce9131435df P 9457d6a3e2e14ca0a3f3ed1e0467480d: Bootstrap starting.
I20240502 14:00:57.377771 6763 tablet_bootstrap.cc:654] T d5a0c9b9d46c469db421cce9131435df P 9457d6a3e2e14ca0a3f3ed1e0467480d: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:57.378345 6763 log.cc:824] T d5a0c9b9d46c469db421cce9131435df P 9457d6a3e2e14ca0a3f3ed1e0467480d: Log is configured to *not* fsync() on all Append() calls
I20240502 14:00:57.379660 6763 tablet_bootstrap.cc:492] T d5a0c9b9d46c469db421cce9131435df P 9457d6a3e2e14ca0a3f3ed1e0467480d: No bootstrap required, opened a new log
I20240502 14:00:57.379793 6763 ts_tablet_manager.cc:1367] T d5a0c9b9d46c469db421cce9131435df P 9457d6a3e2e14ca0a3f3ed1e0467480d: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 14:00:57.382545 6763 raft_consensus.cc:348] T d5a0c9b9d46c469db421cce9131435df P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } }
I20240502 14:00:57.382692 6763 raft_consensus.cc:374] T d5a0c9b9d46c469db421cce9131435df P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:57.382732 6763 raft_consensus.cc:729] T d5a0c9b9d46c469db421cce9131435df P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9457d6a3e2e14ca0a3f3ed1e0467480d, State: Initialized, Role: FOLLOWER
I20240502 14:00:57.383041 6763 consensus_queue.cc:260] T d5a0c9b9d46c469db421cce9131435df P 9457d6a3e2e14ca0a3f3ed1e0467480d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } }
I20240502 14:00:57.383133 6763 raft_consensus.cc:388] T d5a0c9b9d46c469db421cce9131435df P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 14:00:57.383175 6763 raft_consensus.cc:482] T d5a0c9b9d46c469db421cce9131435df P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 14:00:57.383245 6763 raft_consensus.cc:3037] T d5a0c9b9d46c469db421cce9131435df P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:57.383865 6763 raft_consensus.cc:504] T d5a0c9b9d46c469db421cce9131435df P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } }
I20240502 14:00:57.384126 6763 leader_election.cc:302] T d5a0c9b9d46c469db421cce9131435df P 9457d6a3e2e14ca0a3f3ed1e0467480d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9457d6a3e2e14ca0a3f3ed1e0467480d; no voters:
I20240502 14:00:57.384359 6763 leader_election.cc:288] T d5a0c9b9d46c469db421cce9131435df P 9457d6a3e2e14ca0a3f3ed1e0467480d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 14:00:57.384460 6765 raft_consensus.cc:2781] T d5a0c9b9d46c469db421cce9131435df P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:57.384675 6765 raft_consensus.cc:686] T d5a0c9b9d46c469db421cce9131435df P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 1 LEADER]: Becoming Leader. State: Replica: 9457d6a3e2e14ca0a3f3ed1e0467480d, State: Running, Role: LEADER
I20240502 14:00:57.384943 6765 consensus_queue.cc:237] T d5a0c9b9d46c469db421cce9131435df P 9457d6a3e2e14ca0a3f3ed1e0467480d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } }
I20240502 14:00:57.385071 6763 ts_tablet_manager.cc:1398] T d5a0c9b9d46c469db421cce9131435df P 9457d6a3e2e14ca0a3f3ed1e0467480d: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20240502 14:00:57.385277 6763 tablet_bootstrap.cc:492] T 967d1f826a29473184273171be78fdfc P 9457d6a3e2e14ca0a3f3ed1e0467480d: Bootstrap starting.
I20240502 14:00:57.386066 6763 tablet_bootstrap.cc:654] T 967d1f826a29473184273171be78fdfc P 9457d6a3e2e14ca0a3f3ed1e0467480d: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:57.387132 6763 tablet_bootstrap.cc:492] T 967d1f826a29473184273171be78fdfc P 9457d6a3e2e14ca0a3f3ed1e0467480d: No bootstrap required, opened a new log
I20240502 14:00:57.387231 6763 ts_tablet_manager.cc:1367] T 967d1f826a29473184273171be78fdfc P 9457d6a3e2e14ca0a3f3ed1e0467480d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:57.387275 6144 catalog_manager.cc:5484] T d5a0c9b9d46c469db421cce9131435df P 9457d6a3e2e14ca0a3f3ed1e0467480d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9457d6a3e2e14ca0a3f3ed1e0467480d (127.24.66.66). New cstate: current_term: 1 leader_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:57.387735 6763 raft_consensus.cc:348] T 967d1f826a29473184273171be78fdfc P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } }
I20240502 14:00:57.387826 6763 raft_consensus.cc:374] T 967d1f826a29473184273171be78fdfc P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:57.387856 6763 raft_consensus.cc:729] T 967d1f826a29473184273171be78fdfc P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9457d6a3e2e14ca0a3f3ed1e0467480d, State: Initialized, Role: FOLLOWER
I20240502 14:00:57.387969 6763 consensus_queue.cc:260] T 967d1f826a29473184273171be78fdfc P 9457d6a3e2e14ca0a3f3ed1e0467480d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } }
I20240502 14:00:57.388048 6763 raft_consensus.cc:388] T 967d1f826a29473184273171be78fdfc P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 14:00:57.388087 6763 raft_consensus.cc:482] T 967d1f826a29473184273171be78fdfc P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 14:00:57.388136 6763 raft_consensus.cc:3037] T 967d1f826a29473184273171be78fdfc P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:57.389078 6763 raft_consensus.cc:504] T 967d1f826a29473184273171be78fdfc P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } }
I20240502 14:00:57.389170 6763 leader_election.cc:302] T 967d1f826a29473184273171be78fdfc P 9457d6a3e2e14ca0a3f3ed1e0467480d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9457d6a3e2e14ca0a3f3ed1e0467480d; no voters:
I20240502 14:00:57.389235 6763 leader_election.cc:288] T 967d1f826a29473184273171be78fdfc P 9457d6a3e2e14ca0a3f3ed1e0467480d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 14:00:57.389288 6765 raft_consensus.cc:2781] T 967d1f826a29473184273171be78fdfc P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:57.389412 6765 raft_consensus.cc:686] T 967d1f826a29473184273171be78fdfc P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 1 LEADER]: Becoming Leader. State: Replica: 9457d6a3e2e14ca0a3f3ed1e0467480d, State: Running, Role: LEADER
I20240502 14:00:57.389528 6765 consensus_queue.cc:237] T 967d1f826a29473184273171be78fdfc P 9457d6a3e2e14ca0a3f3ed1e0467480d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } }
I20240502 14:00:57.389534 6763 ts_tablet_manager.cc:1398] T 967d1f826a29473184273171be78fdfc P 9457d6a3e2e14ca0a3f3ed1e0467480d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:57.390722 6144 catalog_manager.cc:5484] T 967d1f826a29473184273171be78fdfc P 9457d6a3e2e14ca0a3f3ed1e0467480d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9457d6a3e2e14ca0a3f3ed1e0467480d (127.24.66.66). New cstate: current_term: 1 leader_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:57.549716 6143 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45668:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:57.554495 6266 tablet_service.cc:1467] Processing CreateTablet for tablet 21f51397afd24f51bffcfaea1c3cd257 (DEFAULT_TABLE table=rebalance_test_table_1 [id=30bba5a7c66246bbac6a749cbadd1224]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:57.554543 6397 tablet_service.cc:1467] Processing CreateTablet for tablet 1856266d72e44aa481d2fe099db9a71a (DEFAULT_TABLE table=rebalance_test_table_1 [id=30bba5a7c66246bbac6a749cbadd1224]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:57.554814 6266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21f51397afd24f51bffcfaea1c3cd257. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:57.554847 6397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1856266d72e44aa481d2fe099db9a71a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:57.556506 6756 tablet_bootstrap.cc:492] T 21f51397afd24f51bffcfaea1c3cd257 P e3c8043a08994c0bbf451c44633e2860: Bootstrap starting.
I20240502 14:00:57.556509 6763 tablet_bootstrap.cc:492] T 1856266d72e44aa481d2fe099db9a71a P 9457d6a3e2e14ca0a3f3ed1e0467480d: Bootstrap starting.
I20240502 14:00:57.556872 6266 tablet_service.cc:1467] Processing CreateTablet for tablet 144e42d823ed4ec7be7fe03e18a6254f (DEFAULT_TABLE table=rebalance_test_table_1 [id=30bba5a7c66246bbac6a749cbadd1224]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:57.557086 6266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 144e42d823ed4ec7be7fe03e18a6254f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:57.557457 6763 tablet_bootstrap.cc:654] T 1856266d72e44aa481d2fe099db9a71a P 9457d6a3e2e14ca0a3f3ed1e0467480d: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:57.557498 6756 tablet_bootstrap.cc:654] T 21f51397afd24f51bffcfaea1c3cd257 P e3c8043a08994c0bbf451c44633e2860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:57.558473 6756 tablet_bootstrap.cc:492] T 21f51397afd24f51bffcfaea1c3cd257 P e3c8043a08994c0bbf451c44633e2860: No bootstrap required, opened a new log
I20240502 14:00:57.558563 6756 ts_tablet_manager.cc:1367] T 21f51397afd24f51bffcfaea1c3cd257 P e3c8043a08994c0bbf451c44633e2860: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:57.558921 6763 tablet_bootstrap.cc:492] T 1856266d72e44aa481d2fe099db9a71a P 9457d6a3e2e14ca0a3f3ed1e0467480d: No bootstrap required, opened a new log
I20240502 14:00:57.559002 6763 ts_tablet_manager.cc:1367] T 1856266d72e44aa481d2fe099db9a71a P 9457d6a3e2e14ca0a3f3ed1e0467480d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:57.559005 6756 raft_consensus.cc:348] T 21f51397afd24f51bffcfaea1c3cd257 P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:57.559103 6756 raft_consensus.cc:374] T 21f51397afd24f51bffcfaea1c3cd257 P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:57.559142 6756 raft_consensus.cc:729] T 21f51397afd24f51bffcfaea1c3cd257 P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3c8043a08994c0bbf451c44633e2860, State: Initialized, Role: FOLLOWER
I20240502 14:00:57.559269 6756 consensus_queue.cc:260] T 21f51397afd24f51bffcfaea1c3cd257 P e3c8043a08994c0bbf451c44633e2860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:57.559332 6756 raft_consensus.cc:388] T 21f51397afd24f51bffcfaea1c3cd257 P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 14:00:57.559379 6756 raft_consensus.cc:482] T 21f51397afd24f51bffcfaea1c3cd257 P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 14:00:57.559427 6756 raft_consensus.cc:3037] T 21f51397afd24f51bffcfaea1c3cd257 P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:57.559430 6763 raft_consensus.cc:348] T 1856266d72e44aa481d2fe099db9a71a P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } }
I20240502 14:00:57.559523 6763 raft_consensus.cc:374] T 1856266d72e44aa481d2fe099db9a71a P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:57.559557 6763 raft_consensus.cc:729] T 1856266d72e44aa481d2fe099db9a71a P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9457d6a3e2e14ca0a3f3ed1e0467480d, State: Initialized, Role: FOLLOWER
I20240502 14:00:57.559677 6763 consensus_queue.cc:260] T 1856266d72e44aa481d2fe099db9a71a P 9457d6a3e2e14ca0a3f3ed1e0467480d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } }
I20240502 14:00:57.559770 6763 raft_consensus.cc:388] T 1856266d72e44aa481d2fe099db9a71a P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 14:00:57.559803 6763 raft_consensus.cc:482] T 1856266d72e44aa481d2fe099db9a71a P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 14:00:57.559835 6763 raft_consensus.cc:3037] T 1856266d72e44aa481d2fe099db9a71a P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:57.560276 6756 raft_consensus.cc:504] T 21f51397afd24f51bffcfaea1c3cd257 P e3c8043a08994c0bbf451c44633e2860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:57.560420 6756 leader_election.cc:302] T 21f51397afd24f51bffcfaea1c3cd257 P e3c8043a08994c0bbf451c44633e2860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3c8043a08994c0bbf451c44633e2860; no voters:
I20240502 14:00:57.560515 6756 leader_election.cc:288] T 21f51397afd24f51bffcfaea1c3cd257 P e3c8043a08994c0bbf451c44633e2860 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 14:00:57.560555 6759 raft_consensus.cc:2781] T 21f51397afd24f51bffcfaea1c3cd257 P e3c8043a08994c0bbf451c44633e2860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:57.560635 6759 raft_consensus.cc:686] T 21f51397afd24f51bffcfaea1c3cd257 P e3c8043a08994c0bbf451c44633e2860 [term 1 LEADER]: Becoming Leader. State: Replica: e3c8043a08994c0bbf451c44633e2860, State: Running, Role: LEADER
I20240502 14:00:57.560694 6763 raft_consensus.cc:504] T 1856266d72e44aa481d2fe099db9a71a P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } }
I20240502 14:00:57.560731 6759 consensus_queue.cc:237] T 21f51397afd24f51bffcfaea1c3cd257 P e3c8043a08994c0bbf451c44633e2860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:57.560824 6763 leader_election.cc:302] T 1856266d72e44aa481d2fe099db9a71a P 9457d6a3e2e14ca0a3f3ed1e0467480d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9457d6a3e2e14ca0a3f3ed1e0467480d; no voters:
I20240502 14:00:57.560920 6763 leader_election.cc:288] T 1856266d72e44aa481d2fe099db9a71a P 9457d6a3e2e14ca0a3f3ed1e0467480d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 14:00:57.560956 6765 raft_consensus.cc:2781] T 1856266d72e44aa481d2fe099db9a71a P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:57.561026 6765 raft_consensus.cc:686] T 1856266d72e44aa481d2fe099db9a71a P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 1 LEADER]: Becoming Leader. State: Replica: 9457d6a3e2e14ca0a3f3ed1e0467480d, State: Running, Role: LEADER
I20240502 14:00:57.561125 6765 consensus_queue.cc:237] T 1856266d72e44aa481d2fe099db9a71a P 9457d6a3e2e14ca0a3f3ed1e0467480d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } }
I20240502 14:00:57.561002 6756 ts_tablet_manager.cc:1398] T 21f51397afd24f51bffcfaea1c3cd257 P e3c8043a08994c0bbf451c44633e2860: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240502 14:00:57.561434 6756 tablet_bootstrap.cc:492] T 144e42d823ed4ec7be7fe03e18a6254f P e3c8043a08994c0bbf451c44633e2860: Bootstrap starting.
I20240502 14:00:57.561975 6143 catalog_manager.cc:5484] T 21f51397afd24f51bffcfaea1c3cd257 P e3c8043a08994c0bbf451c44633e2860 reported cstate change: term changed from 0 to 1, leader changed from <none> to e3c8043a08994c0bbf451c44633e2860 (127.24.66.65). New cstate: current_term: 1 leader_uuid: "e3c8043a08994c0bbf451c44633e2860" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:57.562263 6144 catalog_manager.cc:5484] T 1856266d72e44aa481d2fe099db9a71a P 9457d6a3e2e14ca0a3f3ed1e0467480d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9457d6a3e2e14ca0a3f3ed1e0467480d (127.24.66.66). New cstate: current_term: 1 leader_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:57.562649 6756 tablet_bootstrap.cc:654] T 144e42d823ed4ec7be7fe03e18a6254f P e3c8043a08994c0bbf451c44633e2860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:57.563652 6763 ts_tablet_manager.cc:1398] T 1856266d72e44aa481d2fe099db9a71a P 9457d6a3e2e14ca0a3f3ed1e0467480d: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20240502 14:00:57.563843 6756 tablet_bootstrap.cc:492] T 144e42d823ed4ec7be7fe03e18a6254f P e3c8043a08994c0bbf451c44633e2860: No bootstrap required, opened a new log
I20240502 14:00:57.563921 6756 ts_tablet_manager.cc:1367] T 144e42d823ed4ec7be7fe03e18a6254f P e3c8043a08994c0bbf451c44633e2860: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 14:00:57.564410 6756 raft_consensus.cc:348] T 144e42d823ed4ec7be7fe03e18a6254f P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:57.564499 6756 raft_consensus.cc:374] T 144e42d823ed4ec7be7fe03e18a6254f P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:57.564532 6756 raft_consensus.cc:729] T 144e42d823ed4ec7be7fe03e18a6254f P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3c8043a08994c0bbf451c44633e2860, State: Initialized, Role: FOLLOWER
I20240502 14:00:57.564638 6756 consensus_queue.cc:260] T 144e42d823ed4ec7be7fe03e18a6254f P e3c8043a08994c0bbf451c44633e2860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:57.564708 6756 raft_consensus.cc:388] T 144e42d823ed4ec7be7fe03e18a6254f P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 14:00:57.564738 6756 raft_consensus.cc:482] T 144e42d823ed4ec7be7fe03e18a6254f P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 14:00:57.564774 6756 raft_consensus.cc:3037] T 144e42d823ed4ec7be7fe03e18a6254f P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:57.565522 6756 raft_consensus.cc:504] T 144e42d823ed4ec7be7fe03e18a6254f P e3c8043a08994c0bbf451c44633e2860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:57.565631 6756 leader_election.cc:302] T 144e42d823ed4ec7be7fe03e18a6254f P e3c8043a08994c0bbf451c44633e2860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3c8043a08994c0bbf451c44633e2860; no voters:
I20240502 14:00:57.565727 6756 leader_election.cc:288] T 144e42d823ed4ec7be7fe03e18a6254f P e3c8043a08994c0bbf451c44633e2860 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 14:00:57.565917 6756 ts_tablet_manager.cc:1398] T 144e42d823ed4ec7be7fe03e18a6254f P e3c8043a08994c0bbf451c44633e2860: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:57.565973 6760 raft_consensus.cc:2781] T 144e42d823ed4ec7be7fe03e18a6254f P e3c8043a08994c0bbf451c44633e2860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:57.566067 6760 raft_consensus.cc:686] T 144e42d823ed4ec7be7fe03e18a6254f P e3c8043a08994c0bbf451c44633e2860 [term 1 LEADER]: Becoming Leader. State: Replica: e3c8043a08994c0bbf451c44633e2860, State: Running, Role: LEADER
I20240502 14:00:57.566169 6760 consensus_queue.cc:237] T 144e42d823ed4ec7be7fe03e18a6254f P e3c8043a08994c0bbf451c44633e2860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:57.567346 6142 catalog_manager.cc:5484] T 144e42d823ed4ec7be7fe03e18a6254f P e3c8043a08994c0bbf451c44633e2860 reported cstate change: term changed from 0 to 1, leader changed from <none> to e3c8043a08994c0bbf451c44633e2860 (127.24.66.65). New cstate: current_term: 1 leader_uuid: "e3c8043a08994c0bbf451c44633e2860" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:57.719616 6142 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45668:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:57.724457 6396 tablet_service.cc:1467] Processing CreateTablet for tablet 4b9b4636211a4b67a00e5e59cdaec69d (DEFAULT_TABLE table=rebalance_test_table_2 [id=ab660ce4244041879345fbb3ce025c64]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:57.724447 6397 tablet_service.cc:1467] Processing CreateTablet for tablet f50bbd30acb74f37b520a7d680491a88 (DEFAULT_TABLE table=rebalance_test_table_2 [id=ab660ce4244041879345fbb3ce025c64]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:57.724498 6266 tablet_service.cc:1467] Processing CreateTablet for tablet 3f8260387cd6492aac229d76de3816ef (DEFAULT_TABLE table=rebalance_test_table_2 [id=ab660ce4244041879345fbb3ce025c64]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:57.724745 6266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f8260387cd6492aac229d76de3816ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:57.724788 6396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b9b4636211a4b67a00e5e59cdaec69d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:57.724917 6397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f50bbd30acb74f37b520a7d680491a88. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:57.726202 6756 tablet_bootstrap.cc:492] T 3f8260387cd6492aac229d76de3816ef P e3c8043a08994c0bbf451c44633e2860: Bootstrap starting.
I20240502 14:00:57.726850 6763 tablet_bootstrap.cc:492] T 4b9b4636211a4b67a00e5e59cdaec69d P 9457d6a3e2e14ca0a3f3ed1e0467480d: Bootstrap starting.
I20240502 14:00:57.727147 6756 tablet_bootstrap.cc:654] T 3f8260387cd6492aac229d76de3816ef P e3c8043a08994c0bbf451c44633e2860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:57.727771 6763 tablet_bootstrap.cc:654] T 4b9b4636211a4b67a00e5e59cdaec69d P 9457d6a3e2e14ca0a3f3ed1e0467480d: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:57.728130 6756 tablet_bootstrap.cc:492] T 3f8260387cd6492aac229d76de3816ef P e3c8043a08994c0bbf451c44633e2860: No bootstrap required, opened a new log
I20240502 14:00:57.728206 6756 ts_tablet_manager.cc:1367] T 3f8260387cd6492aac229d76de3816ef P e3c8043a08994c0bbf451c44633e2860: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:57.728714 6756 raft_consensus.cc:348] T 3f8260387cd6492aac229d76de3816ef P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:57.728842 6756 raft_consensus.cc:374] T 3f8260387cd6492aac229d76de3816ef P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:57.728875 6756 raft_consensus.cc:729] T 3f8260387cd6492aac229d76de3816ef P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3c8043a08994c0bbf451c44633e2860, State: Initialized, Role: FOLLOWER
I20240502 14:00:57.728919 6763 tablet_bootstrap.cc:492] T 4b9b4636211a4b67a00e5e59cdaec69d P 9457d6a3e2e14ca0a3f3ed1e0467480d: No bootstrap required, opened a new log
I20240502 14:00:57.728987 6763 ts_tablet_manager.cc:1367] T 4b9b4636211a4b67a00e5e59cdaec69d P 9457d6a3e2e14ca0a3f3ed1e0467480d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:57.728987 6756 consensus_queue.cc:260] T 3f8260387cd6492aac229d76de3816ef P e3c8043a08994c0bbf451c44633e2860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:57.729063 6756 raft_consensus.cc:388] T 3f8260387cd6492aac229d76de3816ef P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 14:00:57.729103 6756 raft_consensus.cc:482] T 3f8260387cd6492aac229d76de3816ef P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 14:00:57.729157 6756 raft_consensus.cc:3037] T 3f8260387cd6492aac229d76de3816ef P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:57.729445 6763 raft_consensus.cc:348] T 4b9b4636211a4b67a00e5e59cdaec69d P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } }
I20240502 14:00:57.729540 6763 raft_consensus.cc:374] T 4b9b4636211a4b67a00e5e59cdaec69d P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:57.729596 6763 raft_consensus.cc:729] T 4b9b4636211a4b67a00e5e59cdaec69d P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9457d6a3e2e14ca0a3f3ed1e0467480d, State: Initialized, Role: FOLLOWER
I20240502 14:00:57.729717 6763 consensus_queue.cc:260] T 4b9b4636211a4b67a00e5e59cdaec69d P 9457d6a3e2e14ca0a3f3ed1e0467480d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } }
I20240502 14:00:57.729790 6763 raft_consensus.cc:388] T 4b9b4636211a4b67a00e5e59cdaec69d P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 14:00:57.729832 6763 raft_consensus.cc:482] T 4b9b4636211a4b67a00e5e59cdaec69d P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 14:00:57.729878 6763 raft_consensus.cc:3037] T 4b9b4636211a4b67a00e5e59cdaec69d P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:57.729928 6756 raft_consensus.cc:504] T 3f8260387cd6492aac229d76de3816ef P e3c8043a08994c0bbf451c44633e2860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:57.730047 6756 leader_election.cc:302] T 3f8260387cd6492aac229d76de3816ef P e3c8043a08994c0bbf451c44633e2860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3c8043a08994c0bbf451c44633e2860; no voters:
I20240502 14:00:57.730176 6756 leader_election.cc:288] T 3f8260387cd6492aac229d76de3816ef P e3c8043a08994c0bbf451c44633e2860 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 14:00:57.730240 6758 raft_consensus.cc:2781] T 3f8260387cd6492aac229d76de3816ef P e3c8043a08994c0bbf451c44633e2860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:57.730373 6758 raft_consensus.cc:686] T 3f8260387cd6492aac229d76de3816ef P e3c8043a08994c0bbf451c44633e2860 [term 1 LEADER]: Becoming Leader. State: Replica: e3c8043a08994c0bbf451c44633e2860, State: Running, Role: LEADER
I20240502 14:00:57.730506 6758 consensus_queue.cc:237] T 3f8260387cd6492aac229d76de3816ef P e3c8043a08994c0bbf451c44633e2860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:57.730557 6756 ts_tablet_manager.cc:1398] T 3f8260387cd6492aac229d76de3816ef P e3c8043a08994c0bbf451c44633e2860: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:57.730728 6763 raft_consensus.cc:504] T 4b9b4636211a4b67a00e5e59cdaec69d P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } }
I20240502 14:00:57.730857 6763 leader_election.cc:302] T 4b9b4636211a4b67a00e5e59cdaec69d P 9457d6a3e2e14ca0a3f3ed1e0467480d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9457d6a3e2e14ca0a3f3ed1e0467480d; no voters:
I20240502 14:00:57.730950 6763 leader_election.cc:288] T 4b9b4636211a4b67a00e5e59cdaec69d P 9457d6a3e2e14ca0a3f3ed1e0467480d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 14:00:57.730999 6766 raft_consensus.cc:2781] T 4b9b4636211a4b67a00e5e59cdaec69d P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:57.731073 6766 raft_consensus.cc:686] T 4b9b4636211a4b67a00e5e59cdaec69d P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 1 LEADER]: Becoming Leader. State: Replica: 9457d6a3e2e14ca0a3f3ed1e0467480d, State: Running, Role: LEADER
I20240502 14:00:57.731159 6763 ts_tablet_manager.cc:1398] T 4b9b4636211a4b67a00e5e59cdaec69d P 9457d6a3e2e14ca0a3f3ed1e0467480d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:57.731173 6766 consensus_queue.cc:237] T 4b9b4636211a4b67a00e5e59cdaec69d P 9457d6a3e2e14ca0a3f3ed1e0467480d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } }
I20240502 14:00:57.731285 6763 tablet_bootstrap.cc:492] T f50bbd30acb74f37b520a7d680491a88 P 9457d6a3e2e14ca0a3f3ed1e0467480d: Bootstrap starting.
I20240502 14:00:57.731995 6142 catalog_manager.cc:5484] T 3f8260387cd6492aac229d76de3816ef P e3c8043a08994c0bbf451c44633e2860 reported cstate change: term changed from 0 to 1, leader changed from <none> to e3c8043a08994c0bbf451c44633e2860 (127.24.66.65). New cstate: current_term: 1 leader_uuid: "e3c8043a08994c0bbf451c44633e2860" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:57.732107 6763 tablet_bootstrap.cc:654] T f50bbd30acb74f37b520a7d680491a88 P 9457d6a3e2e14ca0a3f3ed1e0467480d: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:57.732501 6143 catalog_manager.cc:5484] T 4b9b4636211a4b67a00e5e59cdaec69d P 9457d6a3e2e14ca0a3f3ed1e0467480d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9457d6a3e2e14ca0a3f3ed1e0467480d (127.24.66.66). New cstate: current_term: 1 leader_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:57.734160 6763 tablet_bootstrap.cc:492] T f50bbd30acb74f37b520a7d680491a88 P 9457d6a3e2e14ca0a3f3ed1e0467480d: No bootstrap required, opened a new log
I20240502 14:00:57.734223 6763 ts_tablet_manager.cc:1367] T f50bbd30acb74f37b520a7d680491a88 P 9457d6a3e2e14ca0a3f3ed1e0467480d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:57.734619 6763 raft_consensus.cc:348] T f50bbd30acb74f37b520a7d680491a88 P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } }
I20240502 14:00:57.734685 6763 raft_consensus.cc:374] T f50bbd30acb74f37b520a7d680491a88 P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:57.734705 6763 raft_consensus.cc:729] T f50bbd30acb74f37b520a7d680491a88 P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9457d6a3e2e14ca0a3f3ed1e0467480d, State: Initialized, Role: FOLLOWER
I20240502 14:00:57.734789 6763 consensus_queue.cc:260] T f50bbd30acb74f37b520a7d680491a88 P 9457d6a3e2e14ca0a3f3ed1e0467480d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } }
I20240502 14:00:57.734861 6763 raft_consensus.cc:388] T f50bbd30acb74f37b520a7d680491a88 P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 14:00:57.734905 6763 raft_consensus.cc:482] T f50bbd30acb74f37b520a7d680491a88 P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 14:00:57.734941 6763 raft_consensus.cc:3037] T f50bbd30acb74f37b520a7d680491a88 P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:57.735687 6763 raft_consensus.cc:504] T f50bbd30acb74f37b520a7d680491a88 P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } }
I20240502 14:00:57.735800 6763 leader_election.cc:302] T f50bbd30acb74f37b520a7d680491a88 P 9457d6a3e2e14ca0a3f3ed1e0467480d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9457d6a3e2e14ca0a3f3ed1e0467480d; no voters:
I20240502 14:00:57.735913 6763 leader_election.cc:288] T f50bbd30acb74f37b520a7d680491a88 P 9457d6a3e2e14ca0a3f3ed1e0467480d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 14:00:57.736132 6763 ts_tablet_manager.cc:1398] T f50bbd30acb74f37b520a7d680491a88 P 9457d6a3e2e14ca0a3f3ed1e0467480d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:57.736312 6767 raft_consensus.cc:2781] T f50bbd30acb74f37b520a7d680491a88 P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:57.736497 6767 raft_consensus.cc:686] T f50bbd30acb74f37b520a7d680491a88 P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 1 LEADER]: Becoming Leader. State: Replica: 9457d6a3e2e14ca0a3f3ed1e0467480d, State: Running, Role: LEADER
I20240502 14:00:57.736604 6767 consensus_queue.cc:237] T f50bbd30acb74f37b520a7d680491a88 P 9457d6a3e2e14ca0a3f3ed1e0467480d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } }
I20240502 14:00:57.737675 6143 catalog_manager.cc:5484] T f50bbd30acb74f37b520a7d680491a88 P 9457d6a3e2e14ca0a3f3ed1e0467480d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9457d6a3e2e14ca0a3f3ed1e0467480d (127.24.66.66). New cstate: current_term: 1 leader_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:57.884126 6144 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45668:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 14:00:57.888696 6397 tablet_service.cc:1467] Processing CreateTablet for tablet 76b33aca0bd7479689c24201f971f542 (DEFAULT_TABLE table=rebalance_test_table_3 [id=27979787edf1417b97e589ac09867d75]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:57.888703 6266 tablet_service.cc:1467] Processing CreateTablet for tablet 2a1c1f2d98d849b0919fb09408b55355 (DEFAULT_TABLE table=rebalance_test_table_3 [id=27979787edf1417b97e589ac09867d75]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:57.888811 6265 tablet_service.cc:1467] Processing CreateTablet for tablet 6d59de68f3b84d2b8d340aafaccf9338 (DEFAULT_TABLE table=rebalance_test_table_3 [id=27979787edf1417b97e589ac09867d75]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:00:57.888953 6266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a1c1f2d98d849b0919fb09408b55355. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:57.888950 6397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76b33aca0bd7479689c24201f971f542. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:57.889109 6265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d59de68f3b84d2b8d340aafaccf9338. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:57.890537 6756 tablet_bootstrap.cc:492] T 2a1c1f2d98d849b0919fb09408b55355 P e3c8043a08994c0bbf451c44633e2860: Bootstrap starting.
I20240502 14:00:57.890975 6763 tablet_bootstrap.cc:492] T 76b33aca0bd7479689c24201f971f542 P 9457d6a3e2e14ca0a3f3ed1e0467480d: Bootstrap starting.
I20240502 14:00:57.891510 6756 tablet_bootstrap.cc:654] T 2a1c1f2d98d849b0919fb09408b55355 P e3c8043a08994c0bbf451c44633e2860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:57.892076 6763 tablet_bootstrap.cc:654] T 76b33aca0bd7479689c24201f971f542 P 9457d6a3e2e14ca0a3f3ed1e0467480d: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:57.892589 6756 tablet_bootstrap.cc:492] T 2a1c1f2d98d849b0919fb09408b55355 P e3c8043a08994c0bbf451c44633e2860: No bootstrap required, opened a new log
I20240502 14:00:57.892685 6756 ts_tablet_manager.cc:1367] T 2a1c1f2d98d849b0919fb09408b55355 P e3c8043a08994c0bbf451c44633e2860: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:57.893118 6763 tablet_bootstrap.cc:492] T 76b33aca0bd7479689c24201f971f542 P 9457d6a3e2e14ca0a3f3ed1e0467480d: No bootstrap required, opened a new log
I20240502 14:00:57.893137 6756 raft_consensus.cc:348] T 2a1c1f2d98d849b0919fb09408b55355 P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:57.893203 6763 ts_tablet_manager.cc:1367] T 76b33aca0bd7479689c24201f971f542 P 9457d6a3e2e14ca0a3f3ed1e0467480d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:57.893222 6756 raft_consensus.cc:374] T 2a1c1f2d98d849b0919fb09408b55355 P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:57.893250 6756 raft_consensus.cc:729] T 2a1c1f2d98d849b0919fb09408b55355 P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3c8043a08994c0bbf451c44633e2860, State: Initialized, Role: FOLLOWER
I20240502 14:00:57.893374 6756 consensus_queue.cc:260] T 2a1c1f2d98d849b0919fb09408b55355 P e3c8043a08994c0bbf451c44633e2860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:57.893453 6756 raft_consensus.cc:388] T 2a1c1f2d98d849b0919fb09408b55355 P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 14:00:57.893489 6756 raft_consensus.cc:482] T 2a1c1f2d98d849b0919fb09408b55355 P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 14:00:57.893529 6756 raft_consensus.cc:3037] T 2a1c1f2d98d849b0919fb09408b55355 P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:57.893649 6763 raft_consensus.cc:348] T 76b33aca0bd7479689c24201f971f542 P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } }
I20240502 14:00:57.893725 6763 raft_consensus.cc:374] T 76b33aca0bd7479689c24201f971f542 P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:57.893767 6763 raft_consensus.cc:729] T 76b33aca0bd7479689c24201f971f542 P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9457d6a3e2e14ca0a3f3ed1e0467480d, State: Initialized, Role: FOLLOWER
I20240502 14:00:57.893875 6763 consensus_queue.cc:260] T 76b33aca0bd7479689c24201f971f542 P 9457d6a3e2e14ca0a3f3ed1e0467480d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } }
I20240502 14:00:57.893951 6763 raft_consensus.cc:388] T 76b33aca0bd7479689c24201f971f542 P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 14:00:57.893987 6763 raft_consensus.cc:482] T 76b33aca0bd7479689c24201f971f542 P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 14:00:57.894030 6763 raft_consensus.cc:3037] T 76b33aca0bd7479689c24201f971f542 P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:57.894359 6756 raft_consensus.cc:504] T 2a1c1f2d98d849b0919fb09408b55355 P e3c8043a08994c0bbf451c44633e2860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:57.894486 6756 leader_election.cc:302] T 2a1c1f2d98d849b0919fb09408b55355 P e3c8043a08994c0bbf451c44633e2860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3c8043a08994c0bbf451c44633e2860; no voters:
I20240502 14:00:57.894573 6756 leader_election.cc:288] T 2a1c1f2d98d849b0919fb09408b55355 P e3c8043a08994c0bbf451c44633e2860 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 14:00:57.894639 6758 raft_consensus.cc:2781] T 2a1c1f2d98d849b0919fb09408b55355 P e3c8043a08994c0bbf451c44633e2860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:57.894729 6758 raft_consensus.cc:686] T 2a1c1f2d98d849b0919fb09408b55355 P e3c8043a08994c0bbf451c44633e2860 [term 1 LEADER]: Becoming Leader. State: Replica: e3c8043a08994c0bbf451c44633e2860, State: Running, Role: LEADER
I20240502 14:00:57.894846 6756 ts_tablet_manager.cc:1398] T 2a1c1f2d98d849b0919fb09408b55355 P e3c8043a08994c0bbf451c44633e2860: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:57.894841 6758 consensus_queue.cc:237] T 2a1c1f2d98d849b0919fb09408b55355 P e3c8043a08994c0bbf451c44633e2860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:57.895015 6756 tablet_bootstrap.cc:492] T 6d59de68f3b84d2b8d340aafaccf9338 P e3c8043a08994c0bbf451c44633e2860: Bootstrap starting.
I20240502 14:00:57.895082 6763 raft_consensus.cc:504] T 76b33aca0bd7479689c24201f971f542 P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } }
I20240502 14:00:57.895545 6763 leader_election.cc:302] T 76b33aca0bd7479689c24201f971f542 P 9457d6a3e2e14ca0a3f3ed1e0467480d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9457d6a3e2e14ca0a3f3ed1e0467480d; no voters:
I20240502 14:00:57.895669 6763 leader_election.cc:288] T 76b33aca0bd7479689c24201f971f542 P 9457d6a3e2e14ca0a3f3ed1e0467480d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 14:00:57.895720 6766 raft_consensus.cc:2781] T 76b33aca0bd7479689c24201f971f542 P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:57.895808 6766 raft_consensus.cc:686] T 76b33aca0bd7479689c24201f971f542 P 9457d6a3e2e14ca0a3f3ed1e0467480d [term 1 LEADER]: Becoming Leader. State: Replica: 9457d6a3e2e14ca0a3f3ed1e0467480d, State: Running, Role: LEADER
I20240502 14:00:57.895895 6763 ts_tablet_manager.cc:1398] T 76b33aca0bd7479689c24201f971f542 P 9457d6a3e2e14ca0a3f3ed1e0467480d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:57.895905 6766 consensus_queue.cc:237] T 76b33aca0bd7479689c24201f971f542 P 9457d6a3e2e14ca0a3f3ed1e0467480d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } }
I20240502 14:00:57.896238 6756 tablet_bootstrap.cc:654] T 6d59de68f3b84d2b8d340aafaccf9338 P e3c8043a08994c0bbf451c44633e2860: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:57.896191 6144 catalog_manager.cc:5484] T 2a1c1f2d98d849b0919fb09408b55355 P e3c8043a08994c0bbf451c44633e2860 reported cstate change: term changed from 0 to 1, leader changed from <none> to e3c8043a08994c0bbf451c44633e2860 (127.24.66.65). New cstate: current_term: 1 leader_uuid: "e3c8043a08994c0bbf451c44633e2860" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:57.897506 6756 tablet_bootstrap.cc:492] T 6d59de68f3b84d2b8d340aafaccf9338 P e3c8043a08994c0bbf451c44633e2860: No bootstrap required, opened a new log
I20240502 14:00:57.897572 6756 ts_tablet_manager.cc:1367] T 6d59de68f3b84d2b8d340aafaccf9338 P e3c8043a08994c0bbf451c44633e2860: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:00:57.897969 6756 raft_consensus.cc:348] T 6d59de68f3b84d2b8d340aafaccf9338 P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:57.897950 6143 catalog_manager.cc:5484] T 76b33aca0bd7479689c24201f971f542 P 9457d6a3e2e14ca0a3f3ed1e0467480d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9457d6a3e2e14ca0a3f3ed1e0467480d (127.24.66.66). New cstate: current_term: 1 leader_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9457d6a3e2e14ca0a3f3ed1e0467480d" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 42259 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:57.898062 6756 raft_consensus.cc:374] T 6d59de68f3b84d2b8d340aafaccf9338 P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:57.898114 6756 raft_consensus.cc:729] T 6d59de68f3b84d2b8d340aafaccf9338 P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3c8043a08994c0bbf451c44633e2860, State: Initialized, Role: FOLLOWER
I20240502 14:00:57.898231 6756 consensus_queue.cc:260] T 6d59de68f3b84d2b8d340aafaccf9338 P e3c8043a08994c0bbf451c44633e2860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:57.898319 6756 raft_consensus.cc:388] T 6d59de68f3b84d2b8d340aafaccf9338 P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 14:00:57.898357 6756 raft_consensus.cc:482] T 6d59de68f3b84d2b8d340aafaccf9338 P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 14:00:57.898402 6756 raft_consensus.cc:3037] T 6d59de68f3b84d2b8d340aafaccf9338 P e3c8043a08994c0bbf451c44633e2860 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:57.898998 6756 raft_consensus.cc:504] T 6d59de68f3b84d2b8d340aafaccf9338 P e3c8043a08994c0bbf451c44633e2860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:57.899111 6756 leader_election.cc:302] T 6d59de68f3b84d2b8d340aafaccf9338 P e3c8043a08994c0bbf451c44633e2860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3c8043a08994c0bbf451c44633e2860; no voters:
I20240502 14:00:57.899196 6756 leader_election.cc:288] T 6d59de68f3b84d2b8d340aafaccf9338 P e3c8043a08994c0bbf451c44633e2860 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 14:00:57.899240 6759 raft_consensus.cc:2781] T 6d59de68f3b84d2b8d340aafaccf9338 P e3c8043a08994c0bbf451c44633e2860 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:57.899351 6759 raft_consensus.cc:686] T 6d59de68f3b84d2b8d340aafaccf9338 P e3c8043a08994c0bbf451c44633e2860 [term 1 LEADER]: Becoming Leader. State: Replica: e3c8043a08994c0bbf451c44633e2860, State: Running, Role: LEADER
I20240502 14:00:57.899488 6759 consensus_queue.cc:237] T 6d59de68f3b84d2b8d340aafaccf9338 P e3c8043a08994c0bbf451c44633e2860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } }
I20240502 14:00:57.899541 6756 ts_tablet_manager.cc:1398] T 6d59de68f3b84d2b8d340aafaccf9338 P e3c8043a08994c0bbf451c44633e2860: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:57.900876 6143 catalog_manager.cc:5484] T 6d59de68f3b84d2b8d340aafaccf9338 P e3c8043a08994c0bbf451c44633e2860 reported cstate change: term changed from 0 to 1, leader changed from <none> to e3c8043a08994c0bbf451c44633e2860 (127.24.66.65). New cstate: current_term: 1 leader_uuid: "e3c8043a08994c0bbf451c44633e2860" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3c8043a08994c0bbf451c44633e2860" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 42853 } health_report { overall_health: HEALTHY } } }
I20240502 14:00:58.049206 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:41947
--local_ip_for_outbound_sockets=127.24.66.67
--tserver_master_addrs=127.24.66.126:34755
--webserver_port=36963
--webserver_interface=127.24.66.67
--builtin_ntp_servers=127.24.66.84:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 14:00:58.144623 6863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:58.144807 6863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:58.144874 6863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:58.147774 6863 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:58.147826 6863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:58.147913 6863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20240502 14:00:58.151193 6863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:41947
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=36963
--tserver_master_addrs=127.24.66.126:34755
--never_fsync=true
--heap_profile_path=/tmp/kudu.6863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:58.151868 6863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:58.152856 6863 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:58.160562 6868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:58.161170 6869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:58.161805 6871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:58.162401 6863 server_base.cc:1034] running on GCE node
I20240502 14:00:58.162693 6863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:58.163141 6863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:58.164274 6863 hybrid_clock.cc:648] HybridClock initialized: now 1714658458164261 us; error 31 us; skew 500 ppm
I20240502 14:00:58.165730 6863 webserver.cc:460] Webserver started at http://127.24.66.67:36963/ using document root <none> and password file <none>
I20240502 14:00:58.166200 6863 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:58.166273 6863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:58.169149 6863 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 14:00:58.170743 6877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:58.171557 6863 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:58.171654 6863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "3a41f2428d9643b29cd86aff8d3faac7"
format_stamp: "Formatted at 2024-05-02 14:00:53 on dist-test-slave-5npg"
I20240502 14:00:58.171972 6863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:58.190025 6863 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:58.190476 6863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:58.190614 6863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:58.191043 6863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:58.191747 6863 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:58.191792 6863 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:58.191841 6863 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:58.191874 6863 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:58.199460 6863 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:41947
I20240502 14:00:58.199488 6990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:41947 every 8 connection(s)
I20240502 14:00:58.200254 6863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 14:00:58.203967 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 6863
I20240502 14:00:58.204524 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.68:42179
--local_ip_for_outbound_sockets=127.24.66.68
--tserver_master_addrs=127.24.66.126:34755
--webserver_port=43495
--webserver_interface=127.24.66.68
--builtin_ntp_servers=127.24.66.84:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:00:58.208429 6991 heartbeater.cc:347] Connected to a master server at 127.24.66.126:34755
I20240502 14:00:58.208611 6991 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:58.209017 6991 heartbeater.cc:511] Master 127.24.66.126:34755 requested a full tablet report, sending...
I20240502 14:00:58.209951 6143 ts_manager.cc:194] Re-registered known tserver with Master: 3a41f2428d9643b29cd86aff8d3faac7 (127.24.66.67:41947)
I20240502 14:00:58.210530 6143 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.67:59597
W20240502 14:00:58.305908 6994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:58.306071 6994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:58.306108 6994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:58.308912 6994 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:58.308959 6994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:58.309018 6994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.68
I20240502 14:00:58.312260 6994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.68:42179
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.24.66.68
--webserver_port=43495
--tserver_master_addrs=127.24.66.126:34755
--never_fsync=true
--heap_profile_path=/tmp/kudu.6994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.68
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:58.312906 6994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:58.313789 6994 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:58.321477 6999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:58.322113 7000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:58.322799 7002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:58.323513 6994 server_base.cc:1034] running on GCE node
I20240502 14:00:58.323869 6994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:58.324398 6994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:58.325543 6994 hybrid_clock.cc:648] HybridClock initialized: now 1714658458325525 us; error 36 us; skew 500 ppm
I20240502 14:00:58.327062 6994 webserver.cc:460] Webserver started at http://127.24.66.68:43495/ using document root <none> and password file <none>
I20240502 14:00:58.327556 6994 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:58.327653 6994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:58.330780 6994 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:58.332442 7008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:58.333278 6994 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:58.333405 6994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "18f67436b19d4f96aeac62e21dbd0d9e"
format_stamp: "Formatted at 2024-05-02 14:00:53 on dist-test-slave-5npg"
I20240502 14:00:58.333737 6994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:58.358304 6994 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:58.358759 6994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:58.358894 6994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:58.359316 6994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:58.360070 6994 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:58.360117 6994 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:58.360169 6994 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:58.360203 6994 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:58.368130 6994 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.68:42179
I20240502 14:00:58.368170 7121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.68:42179 every 8 connection(s)
I20240502 14:00:58.369065 6994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 14:00:58.370759 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 6994
I20240502 14:00:58.378026 7122 heartbeater.cc:347] Connected to a master server at 127.24.66.126:34755
I20240502 14:00:58.378247 7122 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:58.378860 7122 heartbeater.cc:511] Master 127.24.66.126:34755 requested a full tablet report, sending...
I20240502 14:00:58.379914 6143 ts_manager.cc:194] Re-registered known tserver with Master: 18f67436b19d4f96aeac62e21dbd0d9e (127.24.66.68:42179)
I20240502 14:00:58.380729 6143 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.68:57667
I20240502 14:00:58.544039 6925 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:58.544978 6397 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 14:00:58.545858 6265 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 14:00:58.552595 7056 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:58.645568 7056 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:58.647223 6925 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:58.647279 6397 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 14:00:58.647401 6265 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 14:00:58.728652 7056 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:58.729183 6397 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 14:00:58.729857 6925 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:00:58.737497 6265 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
9286b3b7f4204edabe8203e0862dd75a | 127.24.66.126:34755 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.24.66.84:39677 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+--------------------+---------+----------+----------------+-----------------
18f67436b19d4f96aeac62e21dbd0d9e | 127.24.66.68:42179 | HEALTHY | <none> | 0 | 0
3a41f2428d9643b29cd86aff8d3faac7 | 127.24.66.67:41947 | HEALTHY | <none> | 0 | 0
9457d6a3e2e14ca0a3f3ed1e0467480d | 127.24.66.66:42259 | HEALTHY | <none> | 6 | 0
e3c8043a08994c0bbf451c44633e2860 | 127.24.66.65:42853 | HEALTHY | <none> | 7 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 4 server(s) checked
local_ip_for_outbound_sockets | 127.24.66.65 | experimental | 127.24.66.65:42853
local_ip_for_outbound_sockets | 127.24.66.66 | experimental | 127.24.66.66:42259
local_ip_for_outbound_sockets | 127.24.66.67 | experimental | 127.24.66.67:41947
local_ip_for_outbound_sockets | 127.24.66.68 | experimental | 127.24.66.68:42179
never_fsync | true | unsafe,advanced | all 4 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 4 server(s) checked
raft_prepare_replacement_before_eviction | true | experimental,advanced | all 4 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
server_dump_info_format | pb | hidden | all 4 server(s) checked
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.24.66.65:42853
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.24.66.66:42259
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.24.66.67:41947
server_dump_info_path | /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove_5.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.24.66.68:42179
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+--------------------+-------------------------
builtin_ntp_servers | 127.24.66.84:39677 | all 4 server(s) checked
time_source | builtin | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 6
Third Quartile | 7
Maximum | 7
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 5
Tablets | 13
Replicas | 13
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240502 14:00:58.758347 24841 log_verifier.cc:126] Checking tablet 069c5651bc4c4122942fcbb946f58044
I20240502 14:00:58.765390 24841 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 069c5651bc4c4122942fcbb946f58044
I20240502 14:00:58.765440 24841 log_verifier.cc:126] Checking tablet 144e42d823ed4ec7be7fe03e18a6254f
I20240502 14:00:58.772073 24841 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 144e42d823ed4ec7be7fe03e18a6254f
I20240502 14:00:58.772111 24841 log_verifier.cc:126] Checking tablet 1856266d72e44aa481d2fe099db9a71a
I20240502 14:00:58.778964 24841 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 1856266d72e44aa481d2fe099db9a71a
I20240502 14:00:58.779004 24841 log_verifier.cc:126] Checking tablet 21f51397afd24f51bffcfaea1c3cd257
I20240502 14:00:58.785851 24841 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 21f51397afd24f51bffcfaea1c3cd257
I20240502 14:00:58.785889 24841 log_verifier.cc:126] Checking tablet 2a1c1f2d98d849b0919fb09408b55355
I20240502 14:00:58.792652 24841 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 2a1c1f2d98d849b0919fb09408b55355
I20240502 14:00:58.792688 24841 log_verifier.cc:126] Checking tablet 3f8260387cd6492aac229d76de3816ef
I20240502 14:00:58.799331 24841 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 3f8260387cd6492aac229d76de3816ef
I20240502 14:00:58.799376 24841 log_verifier.cc:126] Checking tablet 4b9b4636211a4b67a00e5e59cdaec69d
I20240502 14:00:58.805955 24841 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 4b9b4636211a4b67a00e5e59cdaec69d
I20240502 14:00:58.805991 24841 log_verifier.cc:126] Checking tablet 6d59de68f3b84d2b8d340aafaccf9338
I20240502 14:00:58.812965 24841 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 6d59de68f3b84d2b8d340aafaccf9338
I20240502 14:00:58.813010 24841 log_verifier.cc:126] Checking tablet 76b33aca0bd7479689c24201f971f542
I20240502 14:00:58.820855 24841 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 76b33aca0bd7479689c24201f971f542
I20240502 14:00:58.820899 24841 log_verifier.cc:126] Checking tablet 967d1f826a29473184273171be78fdfc
I20240502 14:00:58.828786 24841 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 967d1f826a29473184273171be78fdfc
I20240502 14:00:58.828830 24841 log_verifier.cc:126] Checking tablet d5a0c9b9d46c469db421cce9131435df
I20240502 14:00:58.835675 24841 log_verifier.cc:177] Verified matching terms for 1 ops in tablet d5a0c9b9d46c469db421cce9131435df
I20240502 14:00:58.835712 24841 log_verifier.cc:126] Checking tablet ee6a8e1b6e17476fb5cccf3bf28096cd
I20240502 14:00:58.842623 24841 log_verifier.cc:177] Verified matching terms for 1 ops in tablet ee6a8e1b6e17476fb5cccf3bf28096cd
I20240502 14:00:58.842662 24841 log_verifier.cc:126] Checking tablet f50bbd30acb74f37b520a7d680491a88
I20240502 14:00:58.849570 24841 log_verifier.cc:177] Verified matching terms for 2 ops in tablet f50bbd30acb74f37b520a7d680491a88
I20240502 14:00:58.850276 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 6204
I20240502 14:00:58.856609 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 6335
I20240502 14:00:58.862381 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 6863
I20240502 14:00:58.867673 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 6994
I20240502 14:00:58.872490 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 6113
2024-05-02T14:00:58Z chronyd exiting
[ OK ] RebalancerAndSingleReplicaTablets.SingleReplicasStayOrMove/5 (6259 ms)
[----------] 1 test from RebalancerAndSingleReplicaTablets (6259 ms total)
[----------] 2 tests from LocationAwareRebalancingParamTest
[ RUN ] LocationAwareRebalancingParamTest.Rebalance/idx7_rf2_t3_l_1_2_1
I20240502 14:00:58.904099 24841 test_util.cc:268] Using random seed: 140158497
I20240502 14:00:58.904762 24841 ts_itest-base.cc:112] Starting cluster with:
I20240502 14:00:58.904793 24841 ts_itest-base.cc:113] --------------
I20240502 14:00:58.904803 24841 ts_itest-base.cc:114] 4 tablet servers
I20240502 14:00:58.904812 24841 ts_itest-base.cc:115] 2 replicas per TS
I20240502 14:00:58.904821 24841 ts_itest-base.cc:116] --------------
2024-05-02T14:00:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T14:00:58Z Disabled control of system clock
I20240502 14:00:58.912674 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:34949
--webserver_interface=127.24.66.126
--webserver_port=0
--builtin_ntp_servers=127.24.66.84:45171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:34949
--location_mapping_cmd=/tmp/dist-test-taskfX8GOh/build/debug/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/location-assignment.state --map /A:1 --map /B:2 --map /C:1
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false
--master_client_location_assignment_enabled=false
--allow_unsafe_replication_factor with env {}
W20240502 14:00:59.008433 7230 flags.cc:425] Enabled unsafe flag: --allow_unsafe_replication_factor=true
W20240502 14:00:59.008626 7230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:59.008677 7230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:59.008735 7230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:59.011643 7230 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:59.011700 7230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 14:00:59.011725 7230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:59.011742 7230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 14:00:59.011758 7230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 14:00:59.015054 7230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:45171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
--allow_unsafe_replication_factor=true
--enable_range_replica_placement=false
--location_mapping_cmd=/tmp/dist-test-taskfX8GOh/build/debug/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/location-assignment.state --map /A:1 --map /B:2 --map /C:1
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:34949
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:34949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.7230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:59.015733 7230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:59.016778 7230 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:59.023751 7235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:59.024564 7238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:59.025216 7236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:59.025961 7230 server_base.cc:1034] running on GCE node
I20240502 14:00:59.026366 7230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:59.027107 7230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:59.028254 7230 hybrid_clock.cc:648] HybridClock initialized: now 1714658459028245 us; error 22 us; skew 500 ppm
I20240502 14:00:59.029922 7230 webserver.cc:460] Webserver started at http://127.24.66.126:37069/ using document root <none> and password file <none>
I20240502 14:00:59.030383 7230 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:59.030441 7230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:59.030589 7230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:59.031944 7230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "d01bfbc040c04ef68eded7f7765654c2"
format_stamp: "Formatted at 2024-05-02 14:00:59 on dist-test-slave-5npg"
I20240502 14:00:59.032392 7230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "d01bfbc040c04ef68eded7f7765654c2"
format_stamp: "Formatted at 2024-05-02 14:00:59 on dist-test-slave-5npg"
I20240502 14:00:59.035305 7230 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:59.037184 7244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:59.038031 7230 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:59.038141 7230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
uuid: "d01bfbc040c04ef68eded7f7765654c2"
format_stamp: "Formatted at 2024-05-02 14:00:59 on dist-test-slave-5npg"
I20240502 14:00:59.038214 7230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:59.054195 7230 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:59.054677 7230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:59.054809 7230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:59.062780 7230 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:34949
I20240502 14:00:59.062803 7296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:34949 every 8 connection(s)
I20240502 14:00:59.063840 7230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 14:00:59.066581 7297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:00:59.066841 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 7230
I20240502 14:00:59.066977 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 14:00:59.073107 7297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d01bfbc040c04ef68eded7f7765654c2: Bootstrap starting.
I20240502 14:00:59.074996 7297 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d01bfbc040c04ef68eded7f7765654c2: Neither blocks nor log segments found. Creating new log.
I20240502 14:00:59.075785 7297 log.cc:824] T 00000000000000000000000000000000 P d01bfbc040c04ef68eded7f7765654c2: Log is configured to *not* fsync() on all Append() calls
I20240502 14:00:59.077361 7297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d01bfbc040c04ef68eded7f7765654c2: No bootstrap required, opened a new log
I20240502 14:00:59.079653 7297 raft_consensus.cc:348] T 00000000000000000000000000000000 P d01bfbc040c04ef68eded7f7765654c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d01bfbc040c04ef68eded7f7765654c2" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34949 } }
I20240502 14:00:59.079806 7297 raft_consensus.cc:374] T 00000000000000000000000000000000 P d01bfbc040c04ef68eded7f7765654c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:00:59.079870 7297 raft_consensus.cc:729] T 00000000000000000000000000000000 P d01bfbc040c04ef68eded7f7765654c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d01bfbc040c04ef68eded7f7765654c2, State: Initialized, Role: FOLLOWER
I20240502 14:00:59.080231 7297 consensus_queue.cc:260] T 00000000000000000000000000000000 P d01bfbc040c04ef68eded7f7765654c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d01bfbc040c04ef68eded7f7765654c2" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34949 } }
I20240502 14:00:59.080369 7297 raft_consensus.cc:388] T 00000000000000000000000000000000 P d01bfbc040c04ef68eded7f7765654c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 14:00:59.080431 7297 raft_consensus.cc:482] T 00000000000000000000000000000000 P d01bfbc040c04ef68eded7f7765654c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 14:00:59.080531 7297 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d01bfbc040c04ef68eded7f7765654c2 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:00:59.081377 7297 raft_consensus.cc:504] T 00000000000000000000000000000000 P d01bfbc040c04ef68eded7f7765654c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d01bfbc040c04ef68eded7f7765654c2" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34949 } }
I20240502 14:00:59.081691 7297 leader_election.cc:302] T 00000000000000000000000000000000 P d01bfbc040c04ef68eded7f7765654c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d01bfbc040c04ef68eded7f7765654c2; no voters:
I20240502 14:00:59.081969 7297 leader_election.cc:288] T 00000000000000000000000000000000 P d01bfbc040c04ef68eded7f7765654c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 14:00:59.082034 7302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d01bfbc040c04ef68eded7f7765654c2 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:00:59.082263 7302 raft_consensus.cc:686] T 00000000000000000000000000000000 P d01bfbc040c04ef68eded7f7765654c2 [term 1 LEADER]: Becoming Leader. State: Replica: d01bfbc040c04ef68eded7f7765654c2, State: Running, Role: LEADER
I20240502 14:00:59.082862 7302 consensus_queue.cc:237] T 00000000000000000000000000000000 P d01bfbc040c04ef68eded7f7765654c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d01bfbc040c04ef68eded7f7765654c2" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34949 } }
I20240502 14:00:59.082934 7297 sys_catalog.cc:564] T 00000000000000000000000000000000 P d01bfbc040c04ef68eded7f7765654c2 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 14:00:59.084280 7303 sys_catalog.cc:455] T 00000000000000000000000000000000 P d01bfbc040c04ef68eded7f7765654c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d01bfbc040c04ef68eded7f7765654c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d01bfbc040c04ef68eded7f7765654c2" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34949 } } }
I20240502 14:00:59.084430 7303 sys_catalog.cc:458] T 00000000000000000000000000000000 P d01bfbc040c04ef68eded7f7765654c2 [sys.catalog]: This master's current role is: LEADER
I20240502 14:00:59.084671 7304 sys_catalog.cc:455] T 00000000000000000000000000000000 P d01bfbc040c04ef68eded7f7765654c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d01bfbc040c04ef68eded7f7765654c2. Latest consensus state: current_term: 1 leader_uuid: "d01bfbc040c04ef68eded7f7765654c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d01bfbc040c04ef68eded7f7765654c2" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 34949 } } }
I20240502 14:00:59.084767 7304 sys_catalog.cc:458] T 00000000000000000000000000000000 P d01bfbc040c04ef68eded7f7765654c2 [sys.catalog]: This master's current role is: LEADER
I20240502 14:00:59.084865 7311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 14:00:59.088022 7311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 14:00:59.094638 7311 catalog_manager.cc:1347] Generated new cluster ID: b4fbcbc3081e404f97b22f011acb7579
I20240502 14:00:59.094691 7311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 14:00:59.124305 7311 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 14:00:59.125063 7311 catalog_manager.cc:1504] Loading token signing keys...
I20240502 14:00:59.129952 7311 catalog_manager.cc:5857] T 00000000000000000000000000000000 P d01bfbc040c04ef68eded7f7765654c2: Generated new TSK 0
I20240502 14:00:59.130455 7311 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 14:00:59.134291 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:34949
--builtin_ntp_servers=127.24.66.84:45171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 14:00:59.244088 7321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:59.244278 7321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:59.244346 7321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:59.247180 7321 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:59.247237 7321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:59.247321 7321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20240502 14:00:59.250802 7321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:45171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:34949
--never_fsync=true
--heap_profile_path=/tmp/kudu.7321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:59.251543 7321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:59.252578 7321 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:59.260716 7328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:59.261219 7327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:59.261881 7330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:59.262554 7321 server_base.cc:1034] running on GCE node
I20240502 14:00:59.262933 7321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:59.263461 7321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:59.264596 7321 hybrid_clock.cc:648] HybridClock initialized: now 1714658459264586 us; error 22 us; skew 500 ppm
I20240502 14:00:59.266283 7321 webserver.cc:460] Webserver started at http://127.24.66.65:41109/ using document root <none> and password file <none>
I20240502 14:00:59.266777 7321 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:59.266830 7321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:59.267002 7321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:59.268551 7321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "ccb6bf69ed6747ad81147d1566054be5"
format_stamp: "Formatted at 2024-05-02 14:00:59 on dist-test-slave-5npg"
I20240502 14:00:59.268980 7321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "ccb6bf69ed6747ad81147d1566054be5"
format_stamp: "Formatted at 2024-05-02 14:00:59 on dist-test-slave-5npg"
I20240502 14:00:59.271937 7321 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 14:00:59.273861 7336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:59.274755 7321 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20240502 14:00:59.274852 7321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "ccb6bf69ed6747ad81147d1566054be5"
format_stamp: "Formatted at 2024-05-02 14:00:59 on dist-test-slave-5npg"
I20240502 14:00:59.274935 7321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:59.297183 7321 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:59.297708 7321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:59.297852 7321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:59.298332 7321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:59.299121 7321 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:59.299168 7321 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:59.299222 7321 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:59.299261 7321 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:59.306926 7321 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:40249
I20240502 14:00:59.306973 7449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:40249 every 8 connection(s)
I20240502 14:00:59.307747 7321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 14:00:59.310276 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 7321
I20240502 14:00:59.310407 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 14:00:59.312615 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:34949
--builtin_ntp_servers=127.24.66.84:45171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:00:59.316743 7450 heartbeater.cc:347] Connected to a master server at 127.24.66.126:34949
I20240502 14:00:59.316929 7450 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:59.317389 7450 heartbeater.cc:511] Master 127.24.66.126:34949 requested a full tablet report, sending...
I20240502 14:00:59.351121 7261 ts_manager.cc:194] Registered new tserver with Master: ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65:40249)
I20240502 14:00:59.352476 7261 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.65:52613
W20240502 14:00:59.413035 7453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:59.413208 7453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:59.413290 7453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:59.416049 7453 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:59.416108 7453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:59.416198 7453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20240502 14:00:59.419556 7453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:45171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:34949
--never_fsync=true
--heap_profile_path=/tmp/kudu.7453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:59.420213 7453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:59.421123 7453 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:59.428732 7462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:59.429345 7459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:59.429980 7460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:59.430728 7453 server_base.cc:1034] running on GCE node
I20240502 14:00:59.431020 7453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:59.431473 7453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:59.432612 7453 hybrid_clock.cc:648] HybridClock initialized: now 1714658459432599 us; error 25 us; skew 500 ppm
I20240502 14:00:59.434242 7453 webserver.cc:460] Webserver started at http://127.24.66.66:37869/ using document root <none> and password file <none>
I20240502 14:00:59.434693 7453 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:59.434743 7453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:59.434888 7453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:59.436281 7453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "bf1da42196604fc8a3a2732436afd214"
format_stamp: "Formatted at 2024-05-02 14:00:59 on dist-test-slave-5npg"
I20240502 14:00:59.436728 7453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "bf1da42196604fc8a3a2732436afd214"
format_stamp: "Formatted at 2024-05-02 14:00:59 on dist-test-slave-5npg"
I20240502 14:00:59.439724 7453 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:59.441617 7468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:59.442543 7453 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:59.442653 7453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "bf1da42196604fc8a3a2732436afd214"
format_stamp: "Formatted at 2024-05-02 14:00:59 on dist-test-slave-5npg"
I20240502 14:00:59.442739 7453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:59.454635 7453 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:59.455112 7453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:59.455241 7453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:59.455705 7453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:59.456492 7453 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:59.456535 7453 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:59.456576 7453 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:59.456614 7453 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:59.464448 7453 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:38153
I20240502 14:00:59.464488 7581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:38153 every 8 connection(s)
I20240502 14:00:59.465212 7453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 14:00:59.468037 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 7453
I20240502 14:00:59.468144 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 14:00:59.470415 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:34949
--builtin_ntp_servers=127.24.66.84:45171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:00:59.473234 7582 heartbeater.cc:347] Connected to a master server at 127.24.66.126:34949
I20240502 14:00:59.473428 7582 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:59.473999 7582 heartbeater.cc:511] Master 127.24.66.126:34949 requested a full tablet report, sending...
I20240502 14:00:59.506685 7261 ts_manager.cc:194] Registered new tserver with Master: bf1da42196604fc8a3a2732436afd214 (127.24.66.66:38153)
I20240502 14:00:59.507329 7261 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.66:36373
W20240502 14:00:59.569619 7585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:59.569810 7585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:59.569849 7585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:59.572638 7585 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:59.572686 7585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:59.572762 7585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20240502 14:00:59.576085 7585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:45171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:34949
--never_fsync=true
--heap_profile_path=/tmp/kudu.7585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:59.576810 7585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:59.577769 7585 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:59.585511 7594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:59.586108 7591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:59.586723 7592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:59.587374 7585 server_base.cc:1034] running on GCE node
I20240502 14:00:59.587683 7585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:59.588140 7585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:59.589305 7585 hybrid_clock.cc:648] HybridClock initialized: now 1714658459589278 us; error 37 us; skew 500 ppm
I20240502 14:00:59.590986 7585 webserver.cc:460] Webserver started at http://127.24.66.67:35573/ using document root <none> and password file <none>
I20240502 14:00:59.591483 7585 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:59.591539 7585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:59.591709 7585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:59.593168 7585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "9d58e642482142f7b4e27ce750595df7"
format_stamp: "Formatted at 2024-05-02 14:00:59 on dist-test-slave-5npg"
I20240502 14:00:59.593590 7585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "9d58e642482142f7b4e27ce750595df7"
format_stamp: "Formatted at 2024-05-02 14:00:59 on dist-test-slave-5npg"
I20240502 14:00:59.596508 7585 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 14:00:59.598417 7600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:59.599260 7585 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:00:59.599344 7585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "9d58e642482142f7b4e27ce750595df7"
format_stamp: "Formatted at 2024-05-02 14:00:59 on dist-test-slave-5npg"
I20240502 14:00:59.599424 7585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:59.624504 7585 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:59.624970 7585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:59.625104 7585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:59.625538 7585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:59.626371 7585 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:59.626509 7585 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:59.626602 7585 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:59.626638 7585 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:59.634536 7585 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:36913
I20240502 14:00:59.634581 7713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:36913 every 8 connection(s)
I20240502 14:00:59.635298 7585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 14:00:59.636070 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 7585
I20240502 14:00:59.636193 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 14:00:59.638312 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.68:0
--local_ip_for_outbound_sockets=127.24.66.68
--webserver_interface=127.24.66.68
--webserver_port=0
--tserver_master_addrs=127.24.66.126:34949
--builtin_ntp_servers=127.24.66.84:45171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:00:59.643530 7714 heartbeater.cc:347] Connected to a master server at 127.24.66.126:34949
I20240502 14:00:59.643716 7714 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:59.644153 7714 heartbeater.cc:511] Master 127.24.66.126:34949 requested a full tablet report, sending...
I20240502 14:00:59.677134 7261 ts_manager.cc:194] Registered new tserver with Master: 9d58e642482142f7b4e27ce750595df7 (127.24.66.67:36913)
I20240502 14:00:59.677850 7261 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.67:45129
W20240502 14:00:59.736780 7717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:00:59.736961 7717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:00:59.737033 7717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:00:59.739872 7717 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:00:59.739941 7717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:00:59.740041 7717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.68
I20240502 14:00:59.743475 7717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:45171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.24.66.68
--webserver_port=0
--tserver_master_addrs=127.24.66.126:34949
--never_fsync=true
--heap_profile_path=/tmp/kudu.7717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.68
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:00:59.744156 7717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:00:59.745088 7717 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:00:59.752904 7724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:59.753437 7726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:00:59.754024 7723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:00:59.754705 7717 server_base.cc:1034] running on GCE node
I20240502 14:00:59.754999 7717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:00:59.755445 7717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:00:59.756582 7717 hybrid_clock.cc:648] HybridClock initialized: now 1714658459756573 us; error 21 us; skew 500 ppm
I20240502 14:00:59.758167 7717 webserver.cc:460] Webserver started at http://127.24.66.68:36539/ using document root <none> and password file <none>
I20240502 14:00:59.758695 7717 fs_manager.cc:359] Metadata directory not provided
I20240502 14:00:59.758749 7717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:00:59.758911 7717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:00:59.760272 7717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "de464104b5994dcd8e43e854fb53c955"
format_stamp: "Formatted at 2024-05-02 14:00:59 on dist-test-slave-5npg"
I20240502 14:00:59.760725 7717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "de464104b5994dcd8e43e854fb53c955"
format_stamp: "Formatted at 2024-05-02 14:00:59 on dist-test-slave-5npg"
I20240502 14:00:59.763484 7717 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 14:00:59.765321 7732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:59.766132 7717 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 14:00:59.766242 7717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "de464104b5994dcd8e43e854fb53c955"
format_stamp: "Formatted at 2024-05-02 14:00:59 on dist-test-slave-5npg"
I20240502 14:00:59.766331 7717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:00:59.778419 7717 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:00:59.778836 7717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:00:59.778972 7717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:00:59.779389 7717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:00:59.780104 7717 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:00:59.780156 7717 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:59.780210 7717 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:00:59.780238 7717 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:00:59.789719 7717 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.68:43765
I20240502 14:00:59.789762 7845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.68:43765 every 8 connection(s)
I20240502 14:00:59.790675 7717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 14:00:59.794123 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 7717
I20240502 14:00:59.794230 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 14:00:59.799255 7846 heartbeater.cc:347] Connected to a master server at 127.24.66.126:34949
I20240502 14:00:59.799436 7846 heartbeater.cc:464] Registering TS with master...
I20240502 14:00:59.799845 7846 heartbeater.cc:511] Master 127.24.66.126:34949 requested a full tablet report, sending...
I20240502 14:00:59.835134 7261 ts_manager.cc:194] Registered new tserver with Master: de464104b5994dcd8e43e854fb53c955 (127.24.66.68:43765)
I20240502 14:00:59.835760 7261 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.68:55331
I20240502 14:00:59.844130 24841 external_mini_cluster.cc:930] 4 TS(s) registered with all masters
I20240502 14:00:59.852690 24841 external_mini_cluster.cc:930] 4 TS(s) registered with all masters
I20240502 14:00:59.852785 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 7453
I20240502 14:00:59.859372 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 7585
I20240502 14:01:00.354450 7450 heartbeater.cc:503] Master 127.24.66.126:34949 was elected leader, sending a full tablet report...
I20240502 14:01:00.837651 7846 heartbeater.cc:503] Master 127.24.66.126:34949 was elected leader, sending a full tablet report...
I20240502 14:01:03.616708 7261 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34120:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240502 14:01:03.619796 7261 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_0 in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20240502 14:01:03.634248 7778 tablet_service.cc:1467] Processing CreateTablet for tablet 2bb96d00ac544094b41b7f594dda5367 (DEFAULT_TABLE table=rebalance_test_table_0 [id=4beff913099d4bb29ccf75f6ee161e2d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:03.634246 7382 tablet_service.cc:1467] Processing CreateTablet for tablet f276b0640b544d24b21d7c5e18463d20 (DEFAULT_TABLE table=rebalance_test_table_0 [id=4beff913099d4bb29ccf75f6ee161e2d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:03.634248 7780 tablet_service.cc:1467] Processing CreateTablet for tablet 246de49c933c448b814f04b6085e1166 (DEFAULT_TABLE table=rebalance_test_table_0 [id=4beff913099d4bb29ccf75f6ee161e2d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:03.634246 7383 tablet_service.cc:1467] Processing CreateTablet for tablet 246de49c933c448b814f04b6085e1166 (DEFAULT_TABLE table=rebalance_test_table_0 [id=4beff913099d4bb29ccf75f6ee161e2d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:03.634366 7779 tablet_service.cc:1467] Processing CreateTablet for tablet f276b0640b544d24b21d7c5e18463d20 (DEFAULT_TABLE table=rebalance_test_table_0 [id=4beff913099d4bb29ccf75f6ee161e2d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:03.634387 7384 tablet_service.cc:1467] Processing CreateTablet for tablet 2bb96d00ac544094b41b7f594dda5367 (DEFAULT_TABLE table=rebalance_test_table_0 [id=4beff913099d4bb29ccf75f6ee161e2d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:03.635187 7779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f276b0640b544d24b21d7c5e18463d20. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:03.635190 7384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bb96d00ac544094b41b7f594dda5367. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:03.635566 7383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 246de49c933c448b814f04b6085e1166. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:03.635778 7778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bb96d00ac544094b41b7f594dda5367. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:03.636575 7382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f276b0640b544d24b21d7c5e18463d20. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:03.636615 7780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 246de49c933c448b814f04b6085e1166. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:03.641449 7863 tablet_bootstrap.cc:492] T f276b0640b544d24b21d7c5e18463d20 P ccb6bf69ed6747ad81147d1566054be5: Bootstrap starting.
I20240502 14:01:03.642271 7864 tablet_bootstrap.cc:492] T f276b0640b544d24b21d7c5e18463d20 P de464104b5994dcd8e43e854fb53c955: Bootstrap starting.
I20240502 14:01:03.642998 7863 tablet_bootstrap.cc:654] T f276b0640b544d24b21d7c5e18463d20 P ccb6bf69ed6747ad81147d1566054be5: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:03.643643 7863 log.cc:824] T f276b0640b544d24b21d7c5e18463d20 P ccb6bf69ed6747ad81147d1566054be5: Log is configured to *not* fsync() on all Append() calls
I20240502 14:01:03.643704 7864 tablet_bootstrap.cc:654] T f276b0640b544d24b21d7c5e18463d20 P de464104b5994dcd8e43e854fb53c955: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:03.644400 7864 log.cc:824] T f276b0640b544d24b21d7c5e18463d20 P de464104b5994dcd8e43e854fb53c955: Log is configured to *not* fsync() on all Append() calls
I20240502 14:01:03.645099 7863 tablet_bootstrap.cc:492] T f276b0640b544d24b21d7c5e18463d20 P ccb6bf69ed6747ad81147d1566054be5: No bootstrap required, opened a new log
I20240502 14:01:03.645257 7863 ts_tablet_manager.cc:1367] T f276b0640b544d24b21d7c5e18463d20 P ccb6bf69ed6747ad81147d1566054be5: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 14:01:03.645622 7864 tablet_bootstrap.cc:492] T f276b0640b544d24b21d7c5e18463d20 P de464104b5994dcd8e43e854fb53c955: No bootstrap required, opened a new log
I20240502 14:01:03.645756 7864 ts_tablet_manager.cc:1367] T f276b0640b544d24b21d7c5e18463d20 P de464104b5994dcd8e43e854fb53c955: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 14:01:03.647665 7863 raft_consensus.cc:348] T f276b0640b544d24b21d7c5e18463d20 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:03.647830 7863 raft_consensus.cc:374] T f276b0640b544d24b21d7c5e18463d20 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:03.647902 7863 raft_consensus.cc:729] T f276b0640b544d24b21d7c5e18463d20 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccb6bf69ed6747ad81147d1566054be5, State: Initialized, Role: FOLLOWER
I20240502 14:01:03.648285 7863 consensus_queue.cc:260] T f276b0640b544d24b21d7c5e18463d20 P ccb6bf69ed6747ad81147d1566054be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:03.648371 7864 raft_consensus.cc:348] T f276b0640b544d24b21d7c5e18463d20 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:03.648535 7864 raft_consensus.cc:374] T f276b0640b544d24b21d7c5e18463d20 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:03.648591 7864 raft_consensus.cc:729] T f276b0640b544d24b21d7c5e18463d20 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de464104b5994dcd8e43e854fb53c955, State: Initialized, Role: FOLLOWER
I20240502 14:01:03.648949 7864 consensus_queue.cc:260] T f276b0640b544d24b21d7c5e18463d20 P de464104b5994dcd8e43e854fb53c955 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:03.649686 7864 ts_tablet_manager.cc:1398] T f276b0640b544d24b21d7c5e18463d20 P de464104b5994dcd8e43e854fb53c955: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240502 14:01:03.649880 7864 tablet_bootstrap.cc:492] T 2bb96d00ac544094b41b7f594dda5367 P de464104b5994dcd8e43e854fb53c955: Bootstrap starting.
I20240502 14:01:03.650033 7863 ts_tablet_manager.cc:1398] T f276b0640b544d24b21d7c5e18463d20 P ccb6bf69ed6747ad81147d1566054be5: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20240502 14:01:03.650226 7863 tablet_bootstrap.cc:492] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5: Bootstrap starting.
I20240502 14:01:03.650926 7864 tablet_bootstrap.cc:654] T 2bb96d00ac544094b41b7f594dda5367 P de464104b5994dcd8e43e854fb53c955: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:03.651068 7863 tablet_bootstrap.cc:654] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:03.652249 7864 tablet_bootstrap.cc:492] T 2bb96d00ac544094b41b7f594dda5367 P de464104b5994dcd8e43e854fb53c955: No bootstrap required, opened a new log
I20240502 14:01:03.652340 7864 ts_tablet_manager.cc:1367] T 2bb96d00ac544094b41b7f594dda5367 P de464104b5994dcd8e43e854fb53c955: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:03.652822 7864 raft_consensus.cc:348] T 2bb96d00ac544094b41b7f594dda5367 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:03.652904 7864 raft_consensus.cc:374] T 2bb96d00ac544094b41b7f594dda5367 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:03.652942 7864 raft_consensus.cc:729] T 2bb96d00ac544094b41b7f594dda5367 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de464104b5994dcd8e43e854fb53c955, State: Initialized, Role: FOLLOWER
I20240502 14:01:03.653064 7864 consensus_queue.cc:260] T 2bb96d00ac544094b41b7f594dda5367 P de464104b5994dcd8e43e854fb53c955 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:03.653290 7864 ts_tablet_manager.cc:1398] T 2bb96d00ac544094b41b7f594dda5367 P de464104b5994dcd8e43e854fb53c955: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:03.653436 7864 tablet_bootstrap.cc:492] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955: Bootstrap starting.
I20240502 14:01:03.654080 7864 tablet_bootstrap.cc:654] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:03.655023 7863 tablet_bootstrap.cc:492] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5: No bootstrap required, opened a new log
I20240502 14:01:03.655052 7864 tablet_bootstrap.cc:492] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955: No bootstrap required, opened a new log
I20240502 14:01:03.655102 7863 ts_tablet_manager.cc:1367] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20240502 14:01:03.655121 7864 ts_tablet_manager.cc:1367] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:03.655560 7863 raft_consensus.cc:348] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:03.655555 7864 raft_consensus.cc:348] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:03.655638 7864 raft_consensus.cc:374] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:03.655642 7863 raft_consensus.cc:374] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:03.655659 7864 raft_consensus.cc:729] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de464104b5994dcd8e43e854fb53c955, State: Initialized, Role: FOLLOWER
I20240502 14:01:03.655674 7863 raft_consensus.cc:729] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccb6bf69ed6747ad81147d1566054be5, State: Initialized, Role: FOLLOWER
I20240502 14:01:03.655753 7864 consensus_queue.cc:260] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:03.655841 7863 consensus_queue.cc:260] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:03.655994 7864 ts_tablet_manager.cc:1398] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:03.656106 7863 ts_tablet_manager.cc:1398] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 14:01:03.656268 7863 tablet_bootstrap.cc:492] T 246de49c933c448b814f04b6085e1166 P ccb6bf69ed6747ad81147d1566054be5: Bootstrap starting.
I20240502 14:01:03.657084 7863 tablet_bootstrap.cc:654] T 246de49c933c448b814f04b6085e1166 P ccb6bf69ed6747ad81147d1566054be5: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:03.658550 7863 tablet_bootstrap.cc:492] T 246de49c933c448b814f04b6085e1166 P ccb6bf69ed6747ad81147d1566054be5: No bootstrap required, opened a new log
I20240502 14:01:03.658620 7863 ts_tablet_manager.cc:1367] T 246de49c933c448b814f04b6085e1166 P ccb6bf69ed6747ad81147d1566054be5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 14:01:03.658972 7863 raft_consensus.cc:348] T 246de49c933c448b814f04b6085e1166 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:03.659058 7863 raft_consensus.cc:374] T 246de49c933c448b814f04b6085e1166 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:03.659094 7863 raft_consensus.cc:729] T 246de49c933c448b814f04b6085e1166 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccb6bf69ed6747ad81147d1566054be5, State: Initialized, Role: FOLLOWER
I20240502 14:01:03.659206 7863 consensus_queue.cc:260] T 246de49c933c448b814f04b6085e1166 P ccb6bf69ed6747ad81147d1566054be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:03.659442 7863 ts_tablet_manager.cc:1398] T 246de49c933c448b814f04b6085e1166 P ccb6bf69ed6747ad81147d1566054be5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:03.694432 7868 raft_consensus.cc:482] T f276b0640b544d24b21d7c5e18463d20 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:03.694550 7868 raft_consensus.cc:504] T f276b0640b544d24b21d7c5e18463d20 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:03.695273 7868 leader_election.cc:288] T f276b0640b544d24b21d7c5e18463d20 P de464104b5994dcd8e43e854fb53c955 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65:40249)
I20240502 14:01:03.698549 7404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f276b0640b544d24b21d7c5e18463d20" candidate_uuid: "de464104b5994dcd8e43e854fb53c955" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccb6bf69ed6747ad81147d1566054be5" is_pre_election: true
I20240502 14:01:03.698725 7404 raft_consensus.cc:2454] T f276b0640b544d24b21d7c5e18463d20 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de464104b5994dcd8e43e854fb53c955 in term 0.
I20240502 14:01:03.699054 7733 leader_election.cc:302] T f276b0640b544d24b21d7c5e18463d20 P de464104b5994dcd8e43e854fb53c955 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: ccb6bf69ed6747ad81147d1566054be5, de464104b5994dcd8e43e854fb53c955; no voters:
I20240502 14:01:03.699240 7868 raft_consensus.cc:2781] T f276b0640b544d24b21d7c5e18463d20 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:03.699318 7868 raft_consensus.cc:482] T f276b0640b544d24b21d7c5e18463d20 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:03.699347 7868 raft_consensus.cc:3037] T f276b0640b544d24b21d7c5e18463d20 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:03.700047 7868 raft_consensus.cc:504] T f276b0640b544d24b21d7c5e18463d20 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:03.700259 7868 leader_election.cc:288] T f276b0640b544d24b21d7c5e18463d20 P de464104b5994dcd8e43e854fb53c955 [CANDIDATE]: Term 1 election: Requested vote from peers ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65:40249)
I20240502 14:01:03.700587 7404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f276b0640b544d24b21d7c5e18463d20" candidate_uuid: "de464104b5994dcd8e43e854fb53c955" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccb6bf69ed6747ad81147d1566054be5"
I20240502 14:01:03.700671 7404 raft_consensus.cc:3037] T f276b0640b544d24b21d7c5e18463d20 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:03.701687 7404 raft_consensus.cc:2454] T f276b0640b544d24b21d7c5e18463d20 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de464104b5994dcd8e43e854fb53c955 in term 1.
I20240502 14:01:03.701856 7868 raft_consensus.cc:482] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:03.701921 7868 raft_consensus.cc:504] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:03.701951 7733 leader_election.cc:302] T f276b0640b544d24b21d7c5e18463d20 P de464104b5994dcd8e43e854fb53c955 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: ccb6bf69ed6747ad81147d1566054be5, de464104b5994dcd8e43e854fb53c955; no voters:
I20240502 14:01:03.702112 7868 leader_election.cc:288] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65:40249)
I20240502 14:01:03.702216 7874 raft_consensus.cc:2781] T f276b0640b544d24b21d7c5e18463d20 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:03.702319 7874 raft_consensus.cc:686] T f276b0640b544d24b21d7c5e18463d20 P de464104b5994dcd8e43e854fb53c955 [term 1 LEADER]: Becoming Leader. State: Replica: de464104b5994dcd8e43e854fb53c955, State: Running, Role: LEADER
I20240502 14:01:03.702468 7404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "246de49c933c448b814f04b6085e1166" candidate_uuid: "de464104b5994dcd8e43e854fb53c955" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccb6bf69ed6747ad81147d1566054be5" is_pre_election: true
I20240502 14:01:03.702569 7404 raft_consensus.cc:2454] T 246de49c933c448b814f04b6085e1166 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de464104b5994dcd8e43e854fb53c955 in term 0.
I20240502 14:01:03.702560 7874 consensus_queue.cc:237] T f276b0640b544d24b21d7c5e18463d20 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:03.702852 7733 leader_election.cc:302] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: ccb6bf69ed6747ad81147d1566054be5, de464104b5994dcd8e43e854fb53c955; no voters:
I20240502 14:01:03.703110 7868 raft_consensus.cc:2781] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:03.703162 7868 raft_consensus.cc:482] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:03.703202 7868 raft_consensus.cc:3037] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:03.703981 7868 raft_consensus.cc:504] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:03.704218 7868 leader_election.cc:288] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [CANDIDATE]: Term 1 election: Requested vote from peers ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65:40249)
I20240502 14:01:03.704548 7404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "246de49c933c448b814f04b6085e1166" candidate_uuid: "de464104b5994dcd8e43e854fb53c955" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccb6bf69ed6747ad81147d1566054be5"
I20240502 14:01:03.704643 7404 raft_consensus.cc:3037] T 246de49c933c448b814f04b6085e1166 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:03.705334 7404 raft_consensus.cc:2454] T 246de49c933c448b814f04b6085e1166 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de464104b5994dcd8e43e854fb53c955 in term 1.
I20240502 14:01:03.705287 7261 catalog_manager.cc:5484] T f276b0640b544d24b21d7c5e18463d20 P de464104b5994dcd8e43e854fb53c955 reported cstate change: term changed from 0 to 1, leader changed from <none> to de464104b5994dcd8e43e854fb53c955 (127.24.66.68). New cstate: current_term: 1 leader_uuid: "de464104b5994dcd8e43e854fb53c955" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:03.705560 7733 leader_election.cc:302] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: ccb6bf69ed6747ad81147d1566054be5, de464104b5994dcd8e43e854fb53c955; no voters:
I20240502 14:01:03.705709 7868 raft_consensus.cc:2781] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:03.705791 7868 raft_consensus.cc:686] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [term 1 LEADER]: Becoming Leader. State: Replica: de464104b5994dcd8e43e854fb53c955, State: Running, Role: LEADER
I20240502 14:01:03.705888 7868 consensus_queue.cc:237] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:03.708251 7261 catalog_manager.cc:5484] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 reported cstate change: term changed from 0 to 1, leader changed from <none> to de464104b5994dcd8e43e854fb53c955 (127.24.66.68). New cstate: current_term: 1 leader_uuid: "de464104b5994dcd8e43e854fb53c955" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:03.783596 7867 raft_consensus.cc:482] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:03.783689 7867 raft_consensus.cc:504] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:03.784471 7867 leader_election.cc:288] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de464104b5994dcd8e43e854fb53c955 (127.24.66.68:43765)
I20240502 14:01:03.788903 7800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bb96d00ac544094b41b7f594dda5367" candidate_uuid: "ccb6bf69ed6747ad81147d1566054be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de464104b5994dcd8e43e854fb53c955" is_pre_election: true
I20240502 14:01:03.789098 7800 raft_consensus.cc:2454] T 2bb96d00ac544094b41b7f594dda5367 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccb6bf69ed6747ad81147d1566054be5 in term 0.
I20240502 14:01:03.789466 7337 leader_election.cc:302] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: ccb6bf69ed6747ad81147d1566054be5, de464104b5994dcd8e43e854fb53c955; no voters:
I20240502 14:01:03.789714 7867 raft_consensus.cc:2781] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:03.789769 7867 raft_consensus.cc:482] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:03.789816 7867 raft_consensus.cc:3037] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:03.790784 7867 raft_consensus.cc:504] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:03.791036 7867 leader_election.cc:288] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [CANDIDATE]: Term 1 election: Requested vote from peers de464104b5994dcd8e43e854fb53c955 (127.24.66.68:43765)
W20240502 14:01:03.794167 7847 tablet.cc:2369] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 14:01:03.796706 7800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bb96d00ac544094b41b7f594dda5367" candidate_uuid: "ccb6bf69ed6747ad81147d1566054be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de464104b5994dcd8e43e854fb53c955"
I20240502 14:01:03.796808 7800 raft_consensus.cc:3037] T 2bb96d00ac544094b41b7f594dda5367 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:03.797643 7800 raft_consensus.cc:2454] T 2bb96d00ac544094b41b7f594dda5367 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccb6bf69ed6747ad81147d1566054be5 in term 1.
I20240502 14:01:03.797896 7337 leader_election.cc:302] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: ccb6bf69ed6747ad81147d1566054be5, de464104b5994dcd8e43e854fb53c955; no voters:
I20240502 14:01:03.798074 7867 raft_consensus.cc:2781] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:03.798254 7867 raft_consensus.cc:686] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [term 1 LEADER]: Becoming Leader. State: Replica: ccb6bf69ed6747ad81147d1566054be5, State: Running, Role: LEADER
I20240502 14:01:03.798535 7867 consensus_queue.cc:237] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:03.800891 7261 catalog_manager.cc:5484] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65). New cstate: current_term: 1 leader_uuid: "ccb6bf69ed6747ad81147d1566054be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } health_report { overall_health: HEALTHY } } }
W20240502 14:01:03.811247 7451 tablet.cc:2369] T 246de49c933c448b814f04b6085e1166 P ccb6bf69ed6747ad81147d1566054be5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 14:01:03.910173 7880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:03.910346 7880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:03.913151 7880 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240502 14:01:04.002677 7800 raft_consensus.cc:1261] T 2bb96d00ac544094b41b7f594dda5367 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Refusing update from remote peer ccb6bf69ed6747ad81147d1566054be5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:04.003293 7867 consensus_queue.cc:1035] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:04.004685 7404 raft_consensus.cc:1261] T 246de49c933c448b814f04b6085e1166 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Refusing update from remote peer de464104b5994dcd8e43e854fb53c955: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:04.005162 7868 consensus_queue.cc:1035] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:04.009900 7403 raft_consensus.cc:1261] T f276b0640b544d24b21d7c5e18463d20 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Refusing update from remote peer de464104b5994dcd8e43e854fb53c955: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:04.010262 7868 consensus_queue.cc:1035] T f276b0640b544d24b21d7c5e18463d20 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:04.010727 7912 mvcc.cc:205] Tried to move back new op lower bound from 7023241068548878336 to 7023241067720609792. Current Snapshot: MvccSnapshot[applied={T|T < 7023241068548878336}]
I20240502 14:01:04.013826 7909 mvcc.cc:205] Tried to move back new op lower bound from 7023241068567920640 to 7023241067720609792. Current Snapshot: MvccSnapshot[applied={T|T < 7023241068548878336}]
INSERT report
rows total: 2000
time total: 92.0109 ms
time per row: 0.0460054 ms
I20240502 14:01:04.046537 7257 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34120:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240502 14:01:04.046970 7257 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_1 in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20240502 14:01:04.050940 7384 tablet_service.cc:1467] Processing CreateTablet for tablet e79904adb6d54e9884c3113e8354e451 (DEFAULT_TABLE table=rebalance_test_table_1 [id=ef12fb2c44cf4dcfb39f8e343278d5c4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:04.051020 7779 tablet_service.cc:1467] Processing CreateTablet for tablet e79904adb6d54e9884c3113e8354e451 (DEFAULT_TABLE table=rebalance_test_table_1 [id=ef12fb2c44cf4dcfb39f8e343278d5c4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:04.051223 7384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e79904adb6d54e9884c3113e8354e451. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:04.051261 7779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e79904adb6d54e9884c3113e8354e451. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:04.051235 7383 tablet_service.cc:1467] Processing CreateTablet for tablet 6075e58cdba74c919dbb563ff14389ec (DEFAULT_TABLE table=rebalance_test_table_1 [id=ef12fb2c44cf4dcfb39f8e343278d5c4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:04.051467 7383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6075e58cdba74c919dbb563ff14389ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:04.051532 7780 tablet_service.cc:1467] Processing CreateTablet for tablet 6075e58cdba74c919dbb563ff14389ec (DEFAULT_TABLE table=rebalance_test_table_1 [id=ef12fb2c44cf4dcfb39f8e343278d5c4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:04.051717 7780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6075e58cdba74c919dbb563ff14389ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:04.053318 7863 tablet_bootstrap.cc:492] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5: Bootstrap starting.
I20240502 14:01:04.053371 7864 tablet_bootstrap.cc:492] T e79904adb6d54e9884c3113e8354e451 P de464104b5994dcd8e43e854fb53c955: Bootstrap starting.
I20240502 14:01:04.053421 7382 tablet_service.cc:1467] Processing CreateTablet for tablet ba50f31a27794ff3914210f682e2c421 (DEFAULT_TABLE table=rebalance_test_table_1 [id=ef12fb2c44cf4dcfb39f8e343278d5c4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:04.053608 7382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba50f31a27794ff3914210f682e2c421. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:04.053984 7864 tablet_bootstrap.cc:654] T e79904adb6d54e9884c3113e8354e451 P de464104b5994dcd8e43e854fb53c955: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:04.054113 7863 tablet_bootstrap.cc:654] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:04.055094 7863 tablet_bootstrap.cc:492] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5: No bootstrap required, opened a new log
I20240502 14:01:04.055176 7863 ts_tablet_manager.cc:1367] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:01:04.055601 7863 raft_consensus.cc:348] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:04.055629 7779 tablet_service.cc:1467] Processing CreateTablet for tablet ba50f31a27794ff3914210f682e2c421 (DEFAULT_TABLE table=rebalance_test_table_1 [id=ef12fb2c44cf4dcfb39f8e343278d5c4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:04.055687 7863 raft_consensus.cc:374] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:04.055714 7863 raft_consensus.cc:729] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccb6bf69ed6747ad81147d1566054be5, State: Initialized, Role: FOLLOWER
I20240502 14:01:04.055812 7779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba50f31a27794ff3914210f682e2c421. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:04.055814 7863 consensus_queue.cc:260] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:04.056058 7863 ts_tablet_manager.cc:1398] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:04.056193 7863 tablet_bootstrap.cc:492] T e79904adb6d54e9884c3113e8354e451 P ccb6bf69ed6747ad81147d1566054be5: Bootstrap starting.
I20240502 14:01:04.057137 7863 tablet_bootstrap.cc:654] T e79904adb6d54e9884c3113e8354e451 P ccb6bf69ed6747ad81147d1566054be5: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:04.057580 7864 tablet_bootstrap.cc:492] T e79904adb6d54e9884c3113e8354e451 P de464104b5994dcd8e43e854fb53c955: No bootstrap required, opened a new log
I20240502 14:01:04.057658 7864 ts_tablet_manager.cc:1367] T e79904adb6d54e9884c3113e8354e451 P de464104b5994dcd8e43e854fb53c955: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:01:04.058055 7864 raft_consensus.cc:348] T e79904adb6d54e9884c3113e8354e451 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:04.058137 7863 tablet_bootstrap.cc:492] T e79904adb6d54e9884c3113e8354e451 P ccb6bf69ed6747ad81147d1566054be5: No bootstrap required, opened a new log
I20240502 14:01:04.058145 7864 raft_consensus.cc:374] T e79904adb6d54e9884c3113e8354e451 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:04.058178 7864 raft_consensus.cc:729] T e79904adb6d54e9884c3113e8354e451 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de464104b5994dcd8e43e854fb53c955, State: Initialized, Role: FOLLOWER
I20240502 14:01:04.058208 7863 ts_tablet_manager.cc:1367] T e79904adb6d54e9884c3113e8354e451 P ccb6bf69ed6747ad81147d1566054be5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:04.058283 7864 consensus_queue.cc:260] T e79904adb6d54e9884c3113e8354e451 P de464104b5994dcd8e43e854fb53c955 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:04.058547 7864 ts_tablet_manager.cc:1398] T e79904adb6d54e9884c3113e8354e451 P de464104b5994dcd8e43e854fb53c955: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:04.058560 7863 raft_consensus.cc:348] T e79904adb6d54e9884c3113e8354e451 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:04.058632 7863 raft_consensus.cc:374] T e79904adb6d54e9884c3113e8354e451 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:04.058665 7863 raft_consensus.cc:729] T e79904adb6d54e9884c3113e8354e451 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccb6bf69ed6747ad81147d1566054be5, State: Initialized, Role: FOLLOWER
I20240502 14:01:04.058703 7864 tablet_bootstrap.cc:492] T 6075e58cdba74c919dbb563ff14389ec P de464104b5994dcd8e43e854fb53c955: Bootstrap starting.
I20240502 14:01:04.058795 7863 consensus_queue.cc:260] T e79904adb6d54e9884c3113e8354e451 P ccb6bf69ed6747ad81147d1566054be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:04.059330 7863 ts_tablet_manager.cc:1398] T e79904adb6d54e9884c3113e8354e451 P ccb6bf69ed6747ad81147d1566054be5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:04.059442 7864 tablet_bootstrap.cc:654] T 6075e58cdba74c919dbb563ff14389ec P de464104b5994dcd8e43e854fb53c955: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:04.059486 7863 tablet_bootstrap.cc:492] T ba50f31a27794ff3914210f682e2c421 P ccb6bf69ed6747ad81147d1566054be5: Bootstrap starting.
I20240502 14:01:04.060133 7863 tablet_bootstrap.cc:654] T ba50f31a27794ff3914210f682e2c421 P ccb6bf69ed6747ad81147d1566054be5: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:04.060658 7864 tablet_bootstrap.cc:492] T 6075e58cdba74c919dbb563ff14389ec P de464104b5994dcd8e43e854fb53c955: No bootstrap required, opened a new log
I20240502 14:01:04.060748 7864 ts_tablet_manager.cc:1367] T 6075e58cdba74c919dbb563ff14389ec P de464104b5994dcd8e43e854fb53c955: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:04.061245 7864 raft_consensus.cc:348] T 6075e58cdba74c919dbb563ff14389ec P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:04.061333 7864 raft_consensus.cc:374] T 6075e58cdba74c919dbb563ff14389ec P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:04.061373 7863 tablet_bootstrap.cc:492] T ba50f31a27794ff3914210f682e2c421 P ccb6bf69ed6747ad81147d1566054be5: No bootstrap required, opened a new log
I20240502 14:01:04.061380 7864 raft_consensus.cc:729] T 6075e58cdba74c919dbb563ff14389ec P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de464104b5994dcd8e43e854fb53c955, State: Initialized, Role: FOLLOWER
I20240502 14:01:04.061450 7863 ts_tablet_manager.cc:1367] T ba50f31a27794ff3914210f682e2c421 P ccb6bf69ed6747ad81147d1566054be5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:01:04.061483 7864 consensus_queue.cc:260] T 6075e58cdba74c919dbb563ff14389ec P de464104b5994dcd8e43e854fb53c955 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:04.061738 7864 ts_tablet_manager.cc:1398] T 6075e58cdba74c919dbb563ff14389ec P de464104b5994dcd8e43e854fb53c955: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:04.061823 7863 raft_consensus.cc:348] T ba50f31a27794ff3914210f682e2c421 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:04.061880 7864 tablet_bootstrap.cc:492] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955: Bootstrap starting.
I20240502 14:01:04.061898 7863 raft_consensus.cc:374] T ba50f31a27794ff3914210f682e2c421 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:04.062108 7863 raft_consensus.cc:729] T ba50f31a27794ff3914210f682e2c421 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccb6bf69ed6747ad81147d1566054be5, State: Initialized, Role: FOLLOWER
I20240502 14:01:04.062240 7863 consensus_queue.cc:260] T ba50f31a27794ff3914210f682e2c421 P ccb6bf69ed6747ad81147d1566054be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:04.062510 7863 ts_tablet_manager.cc:1398] T ba50f31a27794ff3914210f682e2c421 P ccb6bf69ed6747ad81147d1566054be5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:04.062635 7864 tablet_bootstrap.cc:654] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:04.063592 7864 tablet_bootstrap.cc:492] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955: No bootstrap required, opened a new log
I20240502 14:01:04.063681 7864 ts_tablet_manager.cc:1367] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:04.064010 7864 raft_consensus.cc:348] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:04.064100 7864 raft_consensus.cc:374] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:04.064124 7864 raft_consensus.cc:729] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de464104b5994dcd8e43e854fb53c955, State: Initialized, Role: FOLLOWER
I20240502 14:01:04.064250 7864 consensus_queue.cc:260] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:04.064489 7864 ts_tablet_manager.cc:1398] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:04.195266 7867 raft_consensus.cc:482] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:04.195374 7867 raft_consensus.cc:504] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:04.195731 7867 leader_election.cc:288] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de464104b5994dcd8e43e854fb53c955 (127.24.66.68:43765)
I20240502 14:01:04.196123 7800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6075e58cdba74c919dbb563ff14389ec" candidate_uuid: "ccb6bf69ed6747ad81147d1566054be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de464104b5994dcd8e43e854fb53c955" is_pre_election: true
I20240502 14:01:04.196241 7800 raft_consensus.cc:2454] T 6075e58cdba74c919dbb563ff14389ec P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccb6bf69ed6747ad81147d1566054be5 in term 0.
I20240502 14:01:04.196540 7337 leader_election.cc:302] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: ccb6bf69ed6747ad81147d1566054be5, de464104b5994dcd8e43e854fb53c955; no voters:
I20240502 14:01:04.196727 7867 raft_consensus.cc:2781] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:04.196780 7867 raft_consensus.cc:482] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:04.196830 7867 raft_consensus.cc:3037] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:04.197649 7867 raft_consensus.cc:504] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:04.197894 7867 leader_election.cc:288] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [CANDIDATE]: Term 1 election: Requested vote from peers de464104b5994dcd8e43e854fb53c955 (127.24.66.68:43765)
I20240502 14:01:04.198222 7800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6075e58cdba74c919dbb563ff14389ec" candidate_uuid: "ccb6bf69ed6747ad81147d1566054be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de464104b5994dcd8e43e854fb53c955"
I20240502 14:01:04.198308 7800 raft_consensus.cc:3037] T 6075e58cdba74c919dbb563ff14389ec P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:04.198974 7800 raft_consensus.cc:2454] T 6075e58cdba74c919dbb563ff14389ec P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccb6bf69ed6747ad81147d1566054be5 in term 1.
I20240502 14:01:04.199203 7337 leader_election.cc:302] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: ccb6bf69ed6747ad81147d1566054be5, de464104b5994dcd8e43e854fb53c955; no voters:
I20240502 14:01:04.199342 7867 raft_consensus.cc:2781] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:04.199425 7867 raft_consensus.cc:686] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [term 1 LEADER]: Becoming Leader. State: Replica: ccb6bf69ed6747ad81147d1566054be5, State: Running, Role: LEADER
I20240502 14:01:04.199556 7867 consensus_queue.cc:237] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:04.200853 7257 catalog_manager.cc:5484] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65). New cstate: current_term: 1 leader_uuid: "ccb6bf69ed6747ad81147d1566054be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:04.233906 7874 raft_consensus.cc:482] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:04.233994 7874 raft_consensus.cc:504] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:04.234218 7874 leader_election.cc:288] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65:40249)
I20240502 14:01:04.234580 7403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba50f31a27794ff3914210f682e2c421" candidate_uuid: "de464104b5994dcd8e43e854fb53c955" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccb6bf69ed6747ad81147d1566054be5" is_pre_election: true
I20240502 14:01:04.234702 7403 raft_consensus.cc:2454] T ba50f31a27794ff3914210f682e2c421 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de464104b5994dcd8e43e854fb53c955 in term 0.
I20240502 14:01:04.234942 7733 leader_election.cc:302] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: ccb6bf69ed6747ad81147d1566054be5, de464104b5994dcd8e43e854fb53c955; no voters:
I20240502 14:01:04.235112 7874 raft_consensus.cc:2781] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:04.235159 7874 raft_consensus.cc:482] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:04.235184 7874 raft_consensus.cc:3037] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:04.235946 7874 raft_consensus.cc:504] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:04.236166 7874 leader_election.cc:288] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [CANDIDATE]: Term 1 election: Requested vote from peers ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65:40249)
I20240502 14:01:04.236493 7403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba50f31a27794ff3914210f682e2c421" candidate_uuid: "de464104b5994dcd8e43e854fb53c955" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccb6bf69ed6747ad81147d1566054be5"
I20240502 14:01:04.236609 7403 raft_consensus.cc:3037] T ba50f31a27794ff3914210f682e2c421 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:04.237402 7403 raft_consensus.cc:2454] T ba50f31a27794ff3914210f682e2c421 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de464104b5994dcd8e43e854fb53c955 in term 1.
I20240502 14:01:04.237644 7733 leader_election.cc:302] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: ccb6bf69ed6747ad81147d1566054be5, de464104b5994dcd8e43e854fb53c955; no voters:
I20240502 14:01:04.237807 7874 raft_consensus.cc:2781] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:04.237882 7874 raft_consensus.cc:686] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [term 1 LEADER]: Becoming Leader. State: Replica: de464104b5994dcd8e43e854fb53c955, State: Running, Role: LEADER
I20240502 14:01:04.238008 7874 consensus_queue.cc:237] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:04.239261 7257 catalog_manager.cc:5484] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 reported cstate change: term changed from 0 to 1, leader changed from <none> to de464104b5994dcd8e43e854fb53c955 (127.24.66.68). New cstate: current_term: 1 leader_uuid: "de464104b5994dcd8e43e854fb53c955" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:04.311460 7874 raft_consensus.cc:482] T e79904adb6d54e9884c3113e8354e451 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:04.311591 7874 raft_consensus.cc:504] T e79904adb6d54e9884c3113e8354e451 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:04.311898 7874 leader_election.cc:288] T e79904adb6d54e9884c3113e8354e451 P de464104b5994dcd8e43e854fb53c955 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65:40249)
I20240502 14:01:04.312289 7403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e79904adb6d54e9884c3113e8354e451" candidate_uuid: "de464104b5994dcd8e43e854fb53c955" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccb6bf69ed6747ad81147d1566054be5" is_pre_election: true
I20240502 14:01:04.312455 7403 raft_consensus.cc:2454] T e79904adb6d54e9884c3113e8354e451 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de464104b5994dcd8e43e854fb53c955 in term 0.
I20240502 14:01:04.312734 7733 leader_election.cc:302] T e79904adb6d54e9884c3113e8354e451 P de464104b5994dcd8e43e854fb53c955 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: ccb6bf69ed6747ad81147d1566054be5, de464104b5994dcd8e43e854fb53c955; no voters:
I20240502 14:01:04.312947 7874 raft_consensus.cc:2781] T e79904adb6d54e9884c3113e8354e451 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:04.313004 7874 raft_consensus.cc:482] T e79904adb6d54e9884c3113e8354e451 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:04.313050 7874 raft_consensus.cc:3037] T e79904adb6d54e9884c3113e8354e451 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:04.313800 7874 raft_consensus.cc:504] T e79904adb6d54e9884c3113e8354e451 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:04.314039 7874 leader_election.cc:288] T e79904adb6d54e9884c3113e8354e451 P de464104b5994dcd8e43e854fb53c955 [CANDIDATE]: Term 1 election: Requested vote from peers ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65:40249)
I20240502 14:01:04.314380 7403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e79904adb6d54e9884c3113e8354e451" candidate_uuid: "de464104b5994dcd8e43e854fb53c955" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccb6bf69ed6747ad81147d1566054be5"
I20240502 14:01:04.314477 7403 raft_consensus.cc:3037] T e79904adb6d54e9884c3113e8354e451 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:04.315184 7403 raft_consensus.cc:2454] T e79904adb6d54e9884c3113e8354e451 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de464104b5994dcd8e43e854fb53c955 in term 1.
I20240502 14:01:04.315454 7733 leader_election.cc:302] T e79904adb6d54e9884c3113e8354e451 P de464104b5994dcd8e43e854fb53c955 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: ccb6bf69ed6747ad81147d1566054be5, de464104b5994dcd8e43e854fb53c955; no voters:
I20240502 14:01:04.315663 7874 raft_consensus.cc:2781] T e79904adb6d54e9884c3113e8354e451 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:04.315747 7874 raft_consensus.cc:686] T e79904adb6d54e9884c3113e8354e451 P de464104b5994dcd8e43e854fb53c955 [term 1 LEADER]: Becoming Leader. State: Replica: de464104b5994dcd8e43e854fb53c955, State: Running, Role: LEADER
I20240502 14:01:04.315871 7874 consensus_queue.cc:237] T e79904adb6d54e9884c3113e8354e451 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:04.317374 7257 catalog_manager.cc:5484] T e79904adb6d54e9884c3113e8354e451 P de464104b5994dcd8e43e854fb53c955 reported cstate change: term changed from 0 to 1, leader changed from <none> to de464104b5994dcd8e43e854fb53c955 (127.24.66.68). New cstate: current_term: 1 leader_uuid: "de464104b5994dcd8e43e854fb53c955" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } health_report { overall_health: HEALTHY } } }
W20240502 14:01:04.441442 7934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:04.441627 7934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:04.444487 7934 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240502 14:01:04.508468 7403 raft_consensus.cc:1261] T ba50f31a27794ff3914210f682e2c421 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Refusing update from remote peer de464104b5994dcd8e43e854fb53c955: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:04.508733 7800 raft_consensus.cc:1261] T 6075e58cdba74c919dbb563ff14389ec P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Refusing update from remote peer ccb6bf69ed6747ad81147d1566054be5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:04.508919 7403 raft_consensus.cc:1261] T e79904adb6d54e9884c3113e8354e451 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Refusing update from remote peer de464104b5994dcd8e43e854fb53c955: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:04.508900 7874 consensus_queue.cc:1035] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:04.509091 7867 consensus_queue.cc:1035] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:04.509565 7874 consensus_queue.cc:1035] T e79904adb6d54e9884c3113e8354e451 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 74.4233 ms
time per row: 0.0372116 ms
I20240502 14:01:04.553172 7257 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34120:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240502 14:01:04.553531 7257 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_2 in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20240502 14:01:04.557662 7779 tablet_service.cc:1467] Processing CreateTablet for tablet b3bf9e5109e64e36b7555c6fdd21c504 (DEFAULT_TABLE table=rebalance_test_table_2 [id=6b7ad224eb254c07b5b15e54556a1d33]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:04.557711 7382 tablet_service.cc:1467] Processing CreateTablet for tablet b3bf9e5109e64e36b7555c6fdd21c504 (DEFAULT_TABLE table=rebalance_test_table_2 [id=6b7ad224eb254c07b5b15e54556a1d33]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:04.557930 7779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3bf9e5109e64e36b7555c6fdd21c504. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:04.557932 7382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3bf9e5109e64e36b7555c6fdd21c504. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:04.557905 7384 tablet_service.cc:1467] Processing CreateTablet for tablet 99fabfce542243e48ce1cca51f5089ad (DEFAULT_TABLE table=rebalance_test_table_2 [id=6b7ad224eb254c07b5b15e54556a1d33]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:04.558594 7780 tablet_service.cc:1467] Processing CreateTablet for tablet 99fabfce542243e48ce1cca51f5089ad (DEFAULT_TABLE table=rebalance_test_table_2 [id=6b7ad224eb254c07b5b15e54556a1d33]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:04.558808 7780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99fabfce542243e48ce1cca51f5089ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:04.558930 7778 tablet_service.cc:1467] Processing CreateTablet for tablet c1ef50f64cb94c8bb26dc3c82ebede14 (DEFAULT_TABLE table=rebalance_test_table_2 [id=6b7ad224eb254c07b5b15e54556a1d33]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:04.559116 7778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1ef50f64cb94c8bb26dc3c82ebede14. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:04.560088 7863 tablet_bootstrap.cc:492] T b3bf9e5109e64e36b7555c6fdd21c504 P ccb6bf69ed6747ad81147d1566054be5: Bootstrap starting.
I20240502 14:01:04.560441 7384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99fabfce542243e48ce1cca51f5089ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:04.560670 7864 tablet_bootstrap.cc:492] T 99fabfce542243e48ce1cca51f5089ad P de464104b5994dcd8e43e854fb53c955: Bootstrap starting.
I20240502 14:01:04.560891 7863 tablet_bootstrap.cc:654] T b3bf9e5109e64e36b7555c6fdd21c504 P ccb6bf69ed6747ad81147d1566054be5: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:04.561456 7864 tablet_bootstrap.cc:654] T 99fabfce542243e48ce1cca51f5089ad P de464104b5994dcd8e43e854fb53c955: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:04.562364 7864 tablet_bootstrap.cc:492] T 99fabfce542243e48ce1cca51f5089ad P de464104b5994dcd8e43e854fb53c955: No bootstrap required, opened a new log
I20240502 14:01:04.562458 7864 ts_tablet_manager.cc:1367] T 99fabfce542243e48ce1cca51f5089ad P de464104b5994dcd8e43e854fb53c955: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:04.562619 7863 tablet_bootstrap.cc:492] T b3bf9e5109e64e36b7555c6fdd21c504 P ccb6bf69ed6747ad81147d1566054be5: No bootstrap required, opened a new log
I20240502 14:01:04.562681 7863 ts_tablet_manager.cc:1367] T b3bf9e5109e64e36b7555c6fdd21c504 P ccb6bf69ed6747ad81147d1566054be5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:01:04.562863 7864 raft_consensus.cc:348] T 99fabfce542243e48ce1cca51f5089ad P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:04.562937 7864 raft_consensus.cc:374] T 99fabfce542243e48ce1cca51f5089ad P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:04.562964 7864 raft_consensus.cc:729] T 99fabfce542243e48ce1cca51f5089ad P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de464104b5994dcd8e43e854fb53c955, State: Initialized, Role: FOLLOWER
I20240502 14:01:04.563055 7863 raft_consensus.cc:348] T b3bf9e5109e64e36b7555c6fdd21c504 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:04.563060 7864 consensus_queue.cc:260] T 99fabfce542243e48ce1cca51f5089ad P de464104b5994dcd8e43e854fb53c955 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:04.563123 7863 raft_consensus.cc:374] T b3bf9e5109e64e36b7555c6fdd21c504 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:04.563153 7863 raft_consensus.cc:729] T b3bf9e5109e64e36b7555c6fdd21c504 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccb6bf69ed6747ad81147d1566054be5, State: Initialized, Role: FOLLOWER
I20240502 14:01:04.563247 7863 consensus_queue.cc:260] T b3bf9e5109e64e36b7555c6fdd21c504 P ccb6bf69ed6747ad81147d1566054be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:04.563304 7864 ts_tablet_manager.cc:1398] T 99fabfce542243e48ce1cca51f5089ad P de464104b5994dcd8e43e854fb53c955: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:04.563452 7864 tablet_bootstrap.cc:492] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955: Bootstrap starting.
I20240502 14:01:04.563494 7863 ts_tablet_manager.cc:1398] T b3bf9e5109e64e36b7555c6fdd21c504 P ccb6bf69ed6747ad81147d1566054be5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:04.563637 7863 tablet_bootstrap.cc:492] T 99fabfce542243e48ce1cca51f5089ad P ccb6bf69ed6747ad81147d1566054be5: Bootstrap starting.
I20240502 14:01:04.564188 7864 tablet_bootstrap.cc:654] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:04.564450 7863 tablet_bootstrap.cc:654] T 99fabfce542243e48ce1cca51f5089ad P ccb6bf69ed6747ad81147d1566054be5: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:04.565325 7864 tablet_bootstrap.cc:492] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955: No bootstrap required, opened a new log
I20240502 14:01:04.565451 7864 ts_tablet_manager.cc:1367] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 14:01:04.565528 7863 tablet_bootstrap.cc:492] T 99fabfce542243e48ce1cca51f5089ad P ccb6bf69ed6747ad81147d1566054be5: No bootstrap required, opened a new log
I20240502 14:01:04.565512 7384 tablet_service.cc:1467] Processing CreateTablet for tablet c1ef50f64cb94c8bb26dc3c82ebede14 (DEFAULT_TABLE table=rebalance_test_table_2 [id=6b7ad224eb254c07b5b15e54556a1d33]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:04.565654 7863 ts_tablet_manager.cc:1367] T 99fabfce542243e48ce1cca51f5089ad P ccb6bf69ed6747ad81147d1566054be5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 14:01:04.565770 7384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1ef50f64cb94c8bb26dc3c82ebede14. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:04.566008 7864 raft_consensus.cc:348] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:04.566035 7863 raft_consensus.cc:348] T 99fabfce542243e48ce1cca51f5089ad P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:04.566107 7863 raft_consensus.cc:374] T 99fabfce542243e48ce1cca51f5089ad P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:04.566140 7863 raft_consensus.cc:729] T 99fabfce542243e48ce1cca51f5089ad P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccb6bf69ed6747ad81147d1566054be5, State: Initialized, Role: FOLLOWER
I20240502 14:01:04.566205 7864 raft_consensus.cc:374] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:04.566255 7864 raft_consensus.cc:729] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de464104b5994dcd8e43e854fb53c955, State: Initialized, Role: FOLLOWER
I20240502 14:01:04.566241 7863 consensus_queue.cc:260] T 99fabfce542243e48ce1cca51f5089ad P ccb6bf69ed6747ad81147d1566054be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:04.566370 7864 consensus_queue.cc:260] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:04.566468 7863 ts_tablet_manager.cc:1398] T 99fabfce542243e48ce1cca51f5089ad P ccb6bf69ed6747ad81147d1566054be5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:04.566612 7864 ts_tablet_manager.cc:1398] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:04.566756 7864 tablet_bootstrap.cc:492] T c1ef50f64cb94c8bb26dc3c82ebede14 P de464104b5994dcd8e43e854fb53c955: Bootstrap starting.
I20240502 14:01:04.567427 7863 tablet_bootstrap.cc:492] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5: Bootstrap starting.
I20240502 14:01:04.567526 7864 tablet_bootstrap.cc:654] T c1ef50f64cb94c8bb26dc3c82ebede14 P de464104b5994dcd8e43e854fb53c955: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:04.568270 7863 tablet_bootstrap.cc:654] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:04.568886 7864 tablet_bootstrap.cc:492] T c1ef50f64cb94c8bb26dc3c82ebede14 P de464104b5994dcd8e43e854fb53c955: No bootstrap required, opened a new log
I20240502 14:01:04.569001 7864 ts_tablet_manager.cc:1367] T c1ef50f64cb94c8bb26dc3c82ebede14 P de464104b5994dcd8e43e854fb53c955: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 14:01:04.569213 7863 tablet_bootstrap.cc:492] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5: No bootstrap required, opened a new log
I20240502 14:01:04.569290 7863 ts_tablet_manager.cc:1367] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:04.569522 7864 raft_consensus.cc:348] T c1ef50f64cb94c8bb26dc3c82ebede14 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:04.569676 7864 raft_consensus.cc:374] T c1ef50f64cb94c8bb26dc3c82ebede14 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:04.569670 7863 raft_consensus.cc:348] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:04.569723 7864 raft_consensus.cc:729] T c1ef50f64cb94c8bb26dc3c82ebede14 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de464104b5994dcd8e43e854fb53c955, State: Initialized, Role: FOLLOWER
I20240502 14:01:04.569751 7863 raft_consensus.cc:374] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:04.569790 7863 raft_consensus.cc:729] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccb6bf69ed6747ad81147d1566054be5, State: Initialized, Role: FOLLOWER
I20240502 14:01:04.569831 7864 consensus_queue.cc:260] T c1ef50f64cb94c8bb26dc3c82ebede14 P de464104b5994dcd8e43e854fb53c955 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:04.569881 7863 consensus_queue.cc:260] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:04.570088 7864 ts_tablet_manager.cc:1398] T c1ef50f64cb94c8bb26dc3c82ebede14 P de464104b5994dcd8e43e854fb53c955: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:04.570112 7863 ts_tablet_manager.cc:1398] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:04.646734 7867 raft_consensus.cc:482] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:04.646809 7867 raft_consensus.cc:504] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:04.647022 7867 leader_election.cc:288] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de464104b5994dcd8e43e854fb53c955 (127.24.66.68:43765)
I20240502 14:01:04.647346 7800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ef50f64cb94c8bb26dc3c82ebede14" candidate_uuid: "ccb6bf69ed6747ad81147d1566054be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de464104b5994dcd8e43e854fb53c955" is_pre_election: true
I20240502 14:01:04.647454 7800 raft_consensus.cc:2454] T c1ef50f64cb94c8bb26dc3c82ebede14 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccb6bf69ed6747ad81147d1566054be5 in term 0.
I20240502 14:01:04.647703 7337 leader_election.cc:302] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: ccb6bf69ed6747ad81147d1566054be5, de464104b5994dcd8e43e854fb53c955; no voters:
I20240502 14:01:04.647878 7867 raft_consensus.cc:2781] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:04.647927 7867 raft_consensus.cc:482] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:04.647965 7867 raft_consensus.cc:3037] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:04.648630 7867 raft_consensus.cc:504] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:04.648862 7867 leader_election.cc:288] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [CANDIDATE]: Term 1 election: Requested vote from peers de464104b5994dcd8e43e854fb53c955 (127.24.66.68:43765)
I20240502 14:01:04.649163 7800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ef50f64cb94c8bb26dc3c82ebede14" candidate_uuid: "ccb6bf69ed6747ad81147d1566054be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de464104b5994dcd8e43e854fb53c955"
I20240502 14:01:04.649250 7800 raft_consensus.cc:3037] T c1ef50f64cb94c8bb26dc3c82ebede14 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:04.650236 7800 raft_consensus.cc:2454] T c1ef50f64cb94c8bb26dc3c82ebede14 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccb6bf69ed6747ad81147d1566054be5 in term 1.
I20240502 14:01:04.650480 7337 leader_election.cc:302] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: ccb6bf69ed6747ad81147d1566054be5, de464104b5994dcd8e43e854fb53c955; no voters:
I20240502 14:01:04.650630 7867 raft_consensus.cc:2781] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:04.650704 7867 raft_consensus.cc:686] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [term 1 LEADER]: Becoming Leader. State: Replica: ccb6bf69ed6747ad81147d1566054be5, State: Running, Role: LEADER
I20240502 14:01:04.650827 7867 consensus_queue.cc:237] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:04.651970 7254 catalog_manager.cc:5484] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65). New cstate: current_term: 1 leader_uuid: "ccb6bf69ed6747ad81147d1566054be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:04.690481 7969 raft_consensus.cc:482] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:04.690564 7969 raft_consensus.cc:504] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:04.690794 7969 leader_election.cc:288] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65:40249)
I20240502 14:01:04.691066 7403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3bf9e5109e64e36b7555c6fdd21c504" candidate_uuid: "de464104b5994dcd8e43e854fb53c955" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccb6bf69ed6747ad81147d1566054be5" is_pre_election: true
I20240502 14:01:04.691151 7403 raft_consensus.cc:2454] T b3bf9e5109e64e36b7555c6fdd21c504 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de464104b5994dcd8e43e854fb53c955 in term 0.
I20240502 14:01:04.691375 7733 leader_election.cc:302] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: ccb6bf69ed6747ad81147d1566054be5, de464104b5994dcd8e43e854fb53c955; no voters:
I20240502 14:01:04.691501 7969 raft_consensus.cc:2781] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:04.691548 7969 raft_consensus.cc:482] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:04.691583 7969 raft_consensus.cc:3037] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:04.692198 7969 raft_consensus.cc:504] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:04.692442 7969 leader_election.cc:288] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [CANDIDATE]: Term 1 election: Requested vote from peers ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65:40249)
I20240502 14:01:04.692754 7403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3bf9e5109e64e36b7555c6fdd21c504" candidate_uuid: "de464104b5994dcd8e43e854fb53c955" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccb6bf69ed6747ad81147d1566054be5"
I20240502 14:01:04.692848 7403 raft_consensus.cc:3037] T b3bf9e5109e64e36b7555c6fdd21c504 P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:04.693394 7403 raft_consensus.cc:2454] T b3bf9e5109e64e36b7555c6fdd21c504 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de464104b5994dcd8e43e854fb53c955 in term 1.
I20240502 14:01:04.693637 7733 leader_election.cc:302] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: ccb6bf69ed6747ad81147d1566054be5, de464104b5994dcd8e43e854fb53c955; no voters:
I20240502 14:01:04.693771 7969 raft_consensus.cc:2781] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:04.693846 7969 raft_consensus.cc:686] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [term 1 LEADER]: Becoming Leader. State: Replica: de464104b5994dcd8e43e854fb53c955, State: Running, Role: LEADER
I20240502 14:01:04.693976 7969 consensus_queue.cc:237] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } }
I20240502 14:01:04.695173 7254 catalog_manager.cc:5484] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 reported cstate change: term changed from 0 to 1, leader changed from <none> to de464104b5994dcd8e43e854fb53c955 (127.24.66.68). New cstate: current_term: 1 leader_uuid: "de464104b5994dcd8e43e854fb53c955" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:04.807343 7867 raft_consensus.cc:482] T 99fabfce542243e48ce1cca51f5089ad P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:04.807426 7867 raft_consensus.cc:504] T 99fabfce542243e48ce1cca51f5089ad P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:04.807701 7867 leader_election.cc:288] T 99fabfce542243e48ce1cca51f5089ad P ccb6bf69ed6747ad81147d1566054be5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de464104b5994dcd8e43e854fb53c955 (127.24.66.68:43765)
I20240502 14:01:04.808063 7800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99fabfce542243e48ce1cca51f5089ad" candidate_uuid: "ccb6bf69ed6747ad81147d1566054be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de464104b5994dcd8e43e854fb53c955" is_pre_election: true
I20240502 14:01:04.808157 7800 raft_consensus.cc:2454] T 99fabfce542243e48ce1cca51f5089ad P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccb6bf69ed6747ad81147d1566054be5 in term 0.
I20240502 14:01:04.808465 7337 leader_election.cc:302] T 99fabfce542243e48ce1cca51f5089ad P ccb6bf69ed6747ad81147d1566054be5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: ccb6bf69ed6747ad81147d1566054be5, de464104b5994dcd8e43e854fb53c955; no voters:
I20240502 14:01:04.808604 7867 raft_consensus.cc:2781] T 99fabfce542243e48ce1cca51f5089ad P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:04.808656 7867 raft_consensus.cc:482] T 99fabfce542243e48ce1cca51f5089ad P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:04.808703 7867 raft_consensus.cc:3037] T 99fabfce542243e48ce1cca51f5089ad P ccb6bf69ed6747ad81147d1566054be5 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:04.809273 7867 raft_consensus.cc:504] T 99fabfce542243e48ce1cca51f5089ad P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:04.809468 7867 leader_election.cc:288] T 99fabfce542243e48ce1cca51f5089ad P ccb6bf69ed6747ad81147d1566054be5 [CANDIDATE]: Term 1 election: Requested vote from peers de464104b5994dcd8e43e854fb53c955 (127.24.66.68:43765)
I20240502 14:01:04.809777 7800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99fabfce542243e48ce1cca51f5089ad" candidate_uuid: "ccb6bf69ed6747ad81147d1566054be5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de464104b5994dcd8e43e854fb53c955"
I20240502 14:01:04.809852 7800 raft_consensus.cc:3037] T 99fabfce542243e48ce1cca51f5089ad P de464104b5994dcd8e43e854fb53c955 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:04.810465 7800 raft_consensus.cc:2454] T 99fabfce542243e48ce1cca51f5089ad P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccb6bf69ed6747ad81147d1566054be5 in term 1.
I20240502 14:01:04.810708 7337 leader_election.cc:302] T 99fabfce542243e48ce1cca51f5089ad P ccb6bf69ed6747ad81147d1566054be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: ccb6bf69ed6747ad81147d1566054be5, de464104b5994dcd8e43e854fb53c955; no voters:
I20240502 14:01:04.810873 7867 raft_consensus.cc:2781] T 99fabfce542243e48ce1cca51f5089ad P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:04.810966 7867 raft_consensus.cc:686] T 99fabfce542243e48ce1cca51f5089ad P ccb6bf69ed6747ad81147d1566054be5 [term 1 LEADER]: Becoming Leader. State: Replica: ccb6bf69ed6747ad81147d1566054be5, State: Running, Role: LEADER
I20240502 14:01:04.811087 7867 consensus_queue.cc:237] T 99fabfce542243e48ce1cca51f5089ad P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } }
I20240502 14:01:04.812384 7254 catalog_manager.cc:5484] T 99fabfce542243e48ce1cca51f5089ad P ccb6bf69ed6747ad81147d1566054be5 reported cstate change: term changed from 0 to 1, leader changed from <none> to ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65). New cstate: current_term: 1 leader_uuid: "ccb6bf69ed6747ad81147d1566054be5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } health_report { overall_health: UNKNOWN } } }
W20240502 14:01:04.942359 7981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:04.942575 7981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:04.945528 7981 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240502 14:01:05.007725 7403 raft_consensus.cc:1261] T b3bf9e5109e64e36b7555c6fdd21c504 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Refusing update from remote peer de464104b5994dcd8e43e854fb53c955: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:05.008275 7969 consensus_queue.cc:1035] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:05.009238 7800 raft_consensus.cc:1261] T 99fabfce542243e48ce1cca51f5089ad P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Refusing update from remote peer ccb6bf69ed6747ad81147d1566054be5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:05.009528 7799 raft_consensus.cc:1261] T c1ef50f64cb94c8bb26dc3c82ebede14 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Refusing update from remote peer ccb6bf69ed6747ad81147d1566054be5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:05.010068 7867 consensus_queue.cc:1035] T 99fabfce542243e48ce1cca51f5089ad P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:05.010115 7878 consensus_queue.cc:1035] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 70.9967 ms
time per row: 0.0354984 ms
I20240502 14:01:05.050755 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:38153
--local_ip_for_outbound_sockets=127.24.66.66
--tserver_master_addrs=127.24.66.126:34949
--webserver_port=37869
--webserver_interface=127.24.66.66
--builtin_ntp_servers=127.24.66.84:45171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 14:01:05.148840 8008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:05.149000 8008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:05.149050 8008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:05.151777 8008 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:05.151825 8008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:05.151881 8008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20240502 14:01:05.155267 8008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:45171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:38153
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=37869
--tserver_master_addrs=127.24.66.126:34949
--never_fsync=true
--heap_profile_path=/tmp/kudu.8008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:05.155915 8008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:05.156877 8008 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:05.164711 8013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:05.165292 8016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:05.165971 8014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:05.166558 8008 server_base.cc:1034] running on GCE node
I20240502 14:01:05.166867 8008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:05.167306 8008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:05.168445 8008 hybrid_clock.cc:648] HybridClock initialized: now 1714658465168434 us; error 32 us; skew 500 ppm
I20240502 14:01:05.170097 8008 webserver.cc:460] Webserver started at http://127.24.66.66:37869/ using document root <none> and password file <none>
I20240502 14:01:05.170568 8008 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:05.170640 8008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:05.173449 8008 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20240502 14:01:05.175024 8022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:05.175858 8008 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20240502 14:01:05.175951 8008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "bf1da42196604fc8a3a2732436afd214"
format_stamp: "Formatted at 2024-05-02 14:00:59 on dist-test-slave-5npg"
I20240502 14:01:05.176257 8008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:05.193567 8008 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:05.193970 8008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:05.194101 8008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:05.194505 8008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:05.195191 8008 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:05.195237 8008 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:05.195286 8008 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:05.195318 8008 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:05.202951 8008 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:38153
I20240502 14:01:05.203001 8135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:38153 every 8 connection(s)
I20240502 14:01:05.203841 8008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 14:01:05.205402 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 8008
I20240502 14:01:05.205861 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:36913
--local_ip_for_outbound_sockets=127.24.66.67
--tserver_master_addrs=127.24.66.126:34949
--webserver_port=35573
--webserver_interface=127.24.66.67
--builtin_ntp_servers=127.24.66.84:45171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:01:05.211719 8136 heartbeater.cc:347] Connected to a master server at 127.24.66.126:34949
I20240502 14:01:05.211890 8136 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:05.212291 8136 heartbeater.cc:511] Master 127.24.66.126:34949 requested a full tablet report, sending...
I20240502 14:01:05.213330 7257 ts_manager.cc:194] Re-registered known tserver with Master: bf1da42196604fc8a3a2732436afd214 (127.24.66.66:38153)
I20240502 14:01:05.213841 7257 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.66:46649
W20240502 14:01:05.301828 8139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:05.301982 8139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:05.302042 8139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:05.304744 8139 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:05.304812 8139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:05.304909 8139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20240502 14:01:05.308167 8139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:45171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:36913
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=35573
--tserver_master_addrs=127.24.66.126:34949
--never_fsync=true
--heap_profile_path=/tmp/kudu.8139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:05.308843 8139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:05.309712 8139 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:05.317458 8147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:05.318156 8145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:05.318789 8144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:05.319468 8139 server_base.cc:1034] running on GCE node
I20240502 14:01:05.319779 8139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:05.320216 8139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:05.321362 8139 hybrid_clock.cc:648] HybridClock initialized: now 1714658465321350 us; error 28 us; skew 500 ppm
I20240502 14:01:05.322986 8139 webserver.cc:460] Webserver started at http://127.24.66.67:35573/ using document root <none> and password file <none>
I20240502 14:01:05.323431 8139 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:05.323508 8139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:05.326263 8139 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:05.327937 8153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:05.328837 8139 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:05.328920 8139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "9d58e642482142f7b4e27ce750595df7"
format_stamp: "Formatted at 2024-05-02 14:00:59 on dist-test-slave-5npg"
I20240502 14:01:05.329270 8139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:05.354828 8139 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:05.355242 8139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:05.355399 8139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:05.355837 8139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:05.356551 8139 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:05.356593 8139 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:05.356649 8139 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:05.356688 8139 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:05.364862 8139 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:36913
I20240502 14:01:05.364889 8266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:36913 every 8 connection(s)
I20240502 14:01:05.365752 8139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 14:01:05.371001 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 8139
I20240502 14:01:05.373268 8267 heartbeater.cc:347] Connected to a master server at 127.24.66.126:34949
I20240502 14:01:05.373445 8267 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:05.373852 8267 heartbeater.cc:511] Master 127.24.66.126:34949 requested a full tablet report, sending...
I20240502 14:01:05.374821 7257 ts_manager.cc:194] Re-registered known tserver with Master: 9d58e642482142f7b4e27ce750595df7 (127.24.66.67:36913)
I20240502 14:01:05.375597 7257 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.67:36213
I20240502 14:01:05.539499 7384 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:01:05.540110 8201 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:05.540601 8070 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:05.541116 7778 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:01:05.620254 8201 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:05.620595 8070 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:05.622437 7384 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:01:05.624724 7778 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:01:05.711736 8201 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:05.713380 7778 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:01:05.716517 7384 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:01:05.718554 8070 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:05.781917 8201 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:05.782552 7384 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:01:05.782999 8070 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:05.792575 7778 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:01:05.825685 7799 consensus_queue.cc:237] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: true } }
I20240502 14:01:05.827013 7404 raft_consensus.cc:1261] T b3bf9e5109e64e36b7555c6fdd21c504 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Refusing update from remote peer de464104b5994dcd8e43e854fb53c955: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:05.827482 8408 consensus_queue.cc:1035] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:05.829617 7868 raft_consensus.cc:2932] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 9d58e642482142f7b4e27ce750595df7 (127.24.66.67) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: true } } }
I20240502 14:01:05.830026 7404 raft_consensus.cc:2932] T b3bf9e5109e64e36b7555c6fdd21c504 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 9d58e642482142f7b4e27ce750595df7 (127.24.66.67) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: true } } }
W20240502 14:01:05.831507 7736 consensus_peers.cc:485] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 -> Peer 9d58e642482142f7b4e27ce750595df7 (127.24.66.67:36913): Couldn't send request to peer 9d58e642482142f7b4e27ce750595df7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b3bf9e5109e64e36b7555c6fdd21c504. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:05.832609 7255 catalog_manager.cc:5484] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 reported cstate change: config changed from index -1 to 4, NON_VOTER 9d58e642482142f7b4e27ce750595df7 (127.24.66.67) added. New cstate: current_term: 1 leader_uuid: "de464104b5994dcd8e43e854fb53c955" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:05.838637 7404 consensus_queue.cc:237] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: true } }
I20240502 14:01:05.839915 7799 raft_consensus.cc:1261] T 6075e58cdba74c919dbb563ff14389ec P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Refusing update from remote peer ccb6bf69ed6747ad81147d1566054be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:05.840243 8420 consensus_queue.cc:1035] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:05.841769 7878 raft_consensus.cc:2932] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER bf1da42196604fc8a3a2732436afd214 (127.24.66.66) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: true } } }
I20240502 14:01:05.843101 7799 raft_consensus.cc:2932] T 6075e58cdba74c919dbb563ff14389ec P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER bf1da42196604fc8a3a2732436afd214 (127.24.66.66) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: true } } }
W20240502 14:01:05.843608 7338 consensus_peers.cc:485] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 -> Peer bf1da42196604fc8a3a2732436afd214 (127.24.66.66:38153): Couldn't send request to peer bf1da42196604fc8a3a2732436afd214. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6075e58cdba74c919dbb563ff14389ec. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:05.843889 7257 catalog_manager.cc:5484] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 reported cstate change: config changed from index -1 to 4, NON_VOTER bf1da42196604fc8a3a2732436afd214 (127.24.66.66) added. New cstate: current_term: 1 leader_uuid: "ccb6bf69ed6747ad81147d1566054be5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:05.850291 7799 consensus_queue.cc:237] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: true } }
I20240502 14:01:05.851241 7404 raft_consensus.cc:1261] T 246de49c933c448b814f04b6085e1166 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Refusing update from remote peer de464104b5994dcd8e43e854fb53c955: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:01:05.851533 7736 consensus_peers.cc:485] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 -> Peer 9d58e642482142f7b4e27ce750595df7 (127.24.66.67:36913): Couldn't send request to peer 9d58e642482142f7b4e27ce750595df7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 246de49c933c448b814f04b6085e1166. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:05.851593 7868 consensus_queue.cc:1035] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:05.852921 8408 raft_consensus.cc:2932] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 9d58e642482142f7b4e27ce750595df7 (127.24.66.67) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: true } } }
I20240502 14:01:05.853338 7404 raft_consensus.cc:2932] T 246de49c933c448b814f04b6085e1166 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 9d58e642482142f7b4e27ce750595df7 (127.24.66.67) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: true } } }
I20240502 14:01:05.855016 7255 catalog_manager.cc:5484] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 reported cstate change: config changed from index -1 to 4, NON_VOTER 9d58e642482142f7b4e27ce750595df7 (127.24.66.67) added. New cstate: current_term: 1 leader_uuid: "de464104b5994dcd8e43e854fb53c955" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:05.861053 7404 consensus_queue.cc:237] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: true } }
I20240502 14:01:05.862015 7799 raft_consensus.cc:1261] T 2bb96d00ac544094b41b7f594dda5367 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Refusing update from remote peer ccb6bf69ed6747ad81147d1566054be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:01:05.862219 7338 consensus_peers.cc:485] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 -> Peer bf1da42196604fc8a3a2732436afd214 (127.24.66.66:38153): Couldn't send request to peer bf1da42196604fc8a3a2732436afd214. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2bb96d00ac544094b41b7f594dda5367. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:05.862322 8420 consensus_queue.cc:1035] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:05.863696 7878 raft_consensus.cc:2932] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER bf1da42196604fc8a3a2732436afd214 (127.24.66.66) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: true } } }
I20240502 14:01:05.864010 7799 raft_consensus.cc:2932] T 2bb96d00ac544094b41b7f594dda5367 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER bf1da42196604fc8a3a2732436afd214 (127.24.66.66) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: true } } }
I20240502 14:01:05.865767 7257 catalog_manager.cc:5484] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 reported cstate change: config changed from index -1 to 4, NON_VOTER bf1da42196604fc8a3a2732436afd214 (127.24.66.66) added. New cstate: current_term: 1 leader_uuid: "ccb6bf69ed6747ad81147d1566054be5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:05.871346 7799 consensus_queue.cc:237] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: true } }
I20240502 14:01:05.872231 7404 raft_consensus.cc:1261] T ba50f31a27794ff3914210f682e2c421 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Refusing update from remote peer de464104b5994dcd8e43e854fb53c955: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:01:05.872431 7736 consensus_peers.cc:485] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 -> Peer 9d58e642482142f7b4e27ce750595df7 (127.24.66.67:36913): Couldn't send request to peer 9d58e642482142f7b4e27ce750595df7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ba50f31a27794ff3914210f682e2c421. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:05.872578 8408 consensus_queue.cc:1035] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:05.873852 7868 raft_consensus.cc:2932] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 9d58e642482142f7b4e27ce750595df7 (127.24.66.67) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: true } } }
I20240502 14:01:05.874207 7404 raft_consensus.cc:2932] T ba50f31a27794ff3914210f682e2c421 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER 9d58e642482142f7b4e27ce750595df7 (127.24.66.67) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: true } } }
I20240502 14:01:05.875872 7255 catalog_manager.cc:5484] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 reported cstate change: config changed from index -1 to 4, NON_VOTER 9d58e642482142f7b4e27ce750595df7 (127.24.66.67) added. New cstate: current_term: 1 leader_uuid: "de464104b5994dcd8e43e854fb53c955" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:05.881846 7404 consensus_queue.cc:237] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: true } }
I20240502 14:01:05.882696 7799 raft_consensus.cc:1261] T c1ef50f64cb94c8bb26dc3c82ebede14 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Refusing update from remote peer ccb6bf69ed6747ad81147d1566054be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 14:01:05.882946 7338 consensus_peers.cc:485] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 -> Peer bf1da42196604fc8a3a2732436afd214 (127.24.66.66:38153): Couldn't send request to peer bf1da42196604fc8a3a2732436afd214. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c1ef50f64cb94c8bb26dc3c82ebede14. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:05.883023 8420 consensus_queue.cc:1035] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:05.884264 7878 raft_consensus.cc:2932] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER bf1da42196604fc8a3a2732436afd214 (127.24.66.66) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: true } } }
I20240502 14:01:05.884687 7799 raft_consensus.cc:2932] T c1ef50f64cb94c8bb26dc3c82ebede14 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, NON_VOTER bf1da42196604fc8a3a2732436afd214 (127.24.66.66) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: true } } }
I20240502 14:01:05.887024 7255 catalog_manager.cc:5484] T c1ef50f64cb94c8bb26dc3c82ebede14 P de464104b5994dcd8e43e854fb53c955 reported cstate change: config changed from index -1 to 4, NON_VOTER bf1da42196604fc8a3a2732436afd214 (127.24.66.66) added. New cstate: current_term: 1 leader_uuid: "ccb6bf69ed6747ad81147d1566054be5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: true } } }
I20240502 14:01:06.215464 8136 heartbeater.cc:503] Master 127.24.66.126:34949 was elected leader, sending a full tablet report...
I20240502 14:01:06.289918 8509 ts_tablet_manager.cc:902] T 2bb96d00ac544094b41b7f594dda5367 P bf1da42196604fc8a3a2732436afd214: Initiating tablet copy from peer ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65:40249)
I20240502 14:01:06.290611 8509 tablet_copy_client.cc:308] T 2bb96d00ac544094b41b7f594dda5367 P bf1da42196604fc8a3a2732436afd214: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.65:40249
I20240502 14:01:06.294215 7424 tablet_copy_service.cc:140] P ccb6bf69ed6747ad81147d1566054be5: Received BeginTabletCopySession request for tablet 2bb96d00ac544094b41b7f594dda5367 from peer bf1da42196604fc8a3a2732436afd214 ({username='slave'} at 127.24.66.66:57333)
I20240502 14:01:06.294354 7424 tablet_copy_service.cc:161] P ccb6bf69ed6747ad81147d1566054be5: Beginning new tablet copy session on tablet 2bb96d00ac544094b41b7f594dda5367 from peer bf1da42196604fc8a3a2732436afd214 at {username='slave'} at 127.24.66.66:57333: session id = bf1da42196604fc8a3a2732436afd214-2bb96d00ac544094b41b7f594dda5367
I20240502 14:01:06.295415 7424 tablet_copy_source_session.cc:205] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:06.298492 8509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bb96d00ac544094b41b7f594dda5367. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:06.303655 8509 tablet_copy_client.cc:783] T 2bb96d00ac544094b41b7f594dda5367 P bf1da42196604fc8a3a2732436afd214: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:06.303822 8509 tablet_copy_client.cc:647] T 2bb96d00ac544094b41b7f594dda5367 P bf1da42196604fc8a3a2732436afd214: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:06.304116 8515 ts_tablet_manager.cc:902] T 6075e58cdba74c919dbb563ff14389ec P bf1da42196604fc8a3a2732436afd214: Initiating tablet copy from peer ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65:40249)
I20240502 14:01:06.304662 8515 tablet_copy_client.cc:308] T 6075e58cdba74c919dbb563ff14389ec P bf1da42196604fc8a3a2732436afd214: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.65:40249
I20240502 14:01:06.305074 7423 tablet_copy_service.cc:140] P ccb6bf69ed6747ad81147d1566054be5: Received BeginTabletCopySession request for tablet 6075e58cdba74c919dbb563ff14389ec from peer bf1da42196604fc8a3a2732436afd214 ({username='slave'} at 127.24.66.66:57333)
I20240502 14:01:06.305207 7423 tablet_copy_service.cc:161] P ccb6bf69ed6747ad81147d1566054be5: Beginning new tablet copy session on tablet 6075e58cdba74c919dbb563ff14389ec from peer bf1da42196604fc8a3a2732436afd214 at {username='slave'} at 127.24.66.66:57333: session id = bf1da42196604fc8a3a2732436afd214-6075e58cdba74c919dbb563ff14389ec
I20240502 14:01:06.305482 8509 tablet_copy_client.cc:523] T 2bb96d00ac544094b41b7f594dda5367 P bf1da42196604fc8a3a2732436afd214: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:06.305949 7423 tablet_copy_source_session.cc:205] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:06.306823 8515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6075e58cdba74c919dbb563ff14389ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:06.306988 8509 tablet_bootstrap.cc:492] T 2bb96d00ac544094b41b7f594dda5367 P bf1da42196604fc8a3a2732436afd214: Bootstrap starting.
I20240502 14:01:06.308673 8515 tablet_copy_client.cc:783] T 6075e58cdba74c919dbb563ff14389ec P bf1da42196604fc8a3a2732436afd214: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:06.309929 8515 tablet_copy_client.cc:647] T 6075e58cdba74c919dbb563ff14389ec P bf1da42196604fc8a3a2732436afd214: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:06.311025 8515 tablet_copy_client.cc:523] T 6075e58cdba74c919dbb563ff14389ec P bf1da42196604fc8a3a2732436afd214: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:06.312135 8515 tablet_bootstrap.cc:492] T 6075e58cdba74c919dbb563ff14389ec P bf1da42196604fc8a3a2732436afd214: Bootstrap starting.
I20240502 14:01:06.314188 8509 log.cc:824] T 2bb96d00ac544094b41b7f594dda5367 P bf1da42196604fc8a3a2732436afd214: Log is configured to *not* fsync() on all Append() calls
I20240502 14:01:06.341674 8515 tablet_bootstrap.cc:492] T 6075e58cdba74c919dbb563ff14389ec P bf1da42196604fc8a3a2732436afd214: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:06.341674 8509 tablet_bootstrap.cc:492] T 2bb96d00ac544094b41b7f594dda5367 P bf1da42196604fc8a3a2732436afd214: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:06.342077 8509 tablet_bootstrap.cc:492] T 2bb96d00ac544094b41b7f594dda5367 P bf1da42196604fc8a3a2732436afd214: Bootstrap complete.
I20240502 14:01:06.342078 8515 tablet_bootstrap.cc:492] T 6075e58cdba74c919dbb563ff14389ec P bf1da42196604fc8a3a2732436afd214: Bootstrap complete.
I20240502 14:01:06.342443 8509 ts_tablet_manager.cc:1367] T 2bb96d00ac544094b41b7f594dda5367 P bf1da42196604fc8a3a2732436afd214: Time spent bootstrapping tablet: real 0.036s user 0.029s sys 0.005s
I20240502 14:01:06.342443 8515 ts_tablet_manager.cc:1367] T 6075e58cdba74c919dbb563ff14389ec P bf1da42196604fc8a3a2732436afd214: Time spent bootstrapping tablet: real 0.030s user 0.027s sys 0.004s
I20240502 14:01:06.345098 8509 raft_consensus.cc:348] T 2bb96d00ac544094b41b7f594dda5367 P bf1da42196604fc8a3a2732436afd214 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: true } }
I20240502 14:01:06.345098 8515 raft_consensus.cc:348] T 6075e58cdba74c919dbb563ff14389ec P bf1da42196604fc8a3a2732436afd214 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: true } }
I20240502 14:01:06.345417 8515 raft_consensus.cc:729] T 6075e58cdba74c919dbb563ff14389ec P bf1da42196604fc8a3a2732436afd214 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bf1da42196604fc8a3a2732436afd214, State: Initialized, Role: LEARNER
I20240502 14:01:06.345449 8509 raft_consensus.cc:729] T 2bb96d00ac544094b41b7f594dda5367 P bf1da42196604fc8a3a2732436afd214 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bf1da42196604fc8a3a2732436afd214, State: Initialized, Role: LEARNER
I20240502 14:01:06.345798 8515 consensus_queue.cc:260] T 6075e58cdba74c919dbb563ff14389ec P bf1da42196604fc8a3a2732436afd214 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: true } }
I20240502 14:01:06.345798 8509 consensus_queue.cc:260] T 2bb96d00ac544094b41b7f594dda5367 P bf1da42196604fc8a3a2732436afd214 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: true } }
I20240502 14:01:06.346722 8515 ts_tablet_manager.cc:1398] T 6075e58cdba74c919dbb563ff14389ec P bf1da42196604fc8a3a2732436afd214: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240502 14:01:06.346722 8509 ts_tablet_manager.cc:1398] T 2bb96d00ac544094b41b7f594dda5367 P bf1da42196604fc8a3a2732436afd214: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240502 14:01:06.347461 7424 tablet_copy_service.cc:342] P ccb6bf69ed6747ad81147d1566054be5: Request end of tablet copy session bf1da42196604fc8a3a2732436afd214-6075e58cdba74c919dbb563ff14389ec received from {username='slave'} at 127.24.66.66:57333
I20240502 14:01:06.347559 7424 tablet_copy_service.cc:433] P ccb6bf69ed6747ad81147d1566054be5: ending tablet copy session bf1da42196604fc8a3a2732436afd214-6075e58cdba74c919dbb563ff14389ec on tablet 6075e58cdba74c919dbb563ff14389ec with peer bf1da42196604fc8a3a2732436afd214
I20240502 14:01:06.347847 7423 tablet_copy_service.cc:342] P ccb6bf69ed6747ad81147d1566054be5: Request end of tablet copy session bf1da42196604fc8a3a2732436afd214-2bb96d00ac544094b41b7f594dda5367 received from {username='slave'} at 127.24.66.66:57333
I20240502 14:01:06.347915 7423 tablet_copy_service.cc:433] P ccb6bf69ed6747ad81147d1566054be5: ending tablet copy session bf1da42196604fc8a3a2732436afd214-2bb96d00ac544094b41b7f594dda5367 on tablet 2bb96d00ac544094b41b7f594dda5367 with peer bf1da42196604fc8a3a2732436afd214
I20240502 14:01:06.349630 8529 ts_tablet_manager.cc:902] T b3bf9e5109e64e36b7555c6fdd21c504 P 9d58e642482142f7b4e27ce750595df7: Initiating tablet copy from peer de464104b5994dcd8e43e854fb53c955 (127.24.66.68:43765)
I20240502 14:01:06.350406 8529 tablet_copy_client.cc:308] T b3bf9e5109e64e36b7555c6fdd21c504 P 9d58e642482142f7b4e27ce750595df7: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:43765
I20240502 14:01:06.354892 7817 tablet_copy_service.cc:140] P de464104b5994dcd8e43e854fb53c955: Received BeginTabletCopySession request for tablet b3bf9e5109e64e36b7555c6fdd21c504 from peer 9d58e642482142f7b4e27ce750595df7 ({username='slave'} at 127.24.66.67:48763)
I20240502 14:01:06.355018 7817 tablet_copy_service.cc:161] P de464104b5994dcd8e43e854fb53c955: Beginning new tablet copy session on tablet b3bf9e5109e64e36b7555c6fdd21c504 from peer 9d58e642482142f7b4e27ce750595df7 at {username='slave'} at 127.24.66.67:48763: session id = 9d58e642482142f7b4e27ce750595df7-b3bf9e5109e64e36b7555c6fdd21c504
I20240502 14:01:06.356365 7817 tablet_copy_source_session.cc:205] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:06.360049 8529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3bf9e5109e64e36b7555c6fdd21c504. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:06.365833 8529 tablet_copy_client.cc:783] T b3bf9e5109e64e36b7555c6fdd21c504 P 9d58e642482142f7b4e27ce750595df7: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:06.366029 8529 tablet_copy_client.cc:647] T b3bf9e5109e64e36b7555c6fdd21c504 P 9d58e642482142f7b4e27ce750595df7: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:06.367585 8529 tablet_copy_client.cc:523] T b3bf9e5109e64e36b7555c6fdd21c504 P 9d58e642482142f7b4e27ce750595df7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:06.369328 8529 tablet_bootstrap.cc:492] T b3bf9e5109e64e36b7555c6fdd21c504 P 9d58e642482142f7b4e27ce750595df7: Bootstrap starting.
I20240502 14:01:06.376446 8529 log.cc:824] T b3bf9e5109e64e36b7555c6fdd21c504 P 9d58e642482142f7b4e27ce750595df7: Log is configured to *not* fsync() on all Append() calls
I20240502 14:01:06.377207 8267 heartbeater.cc:503] Master 127.24.66.126:34949 was elected leader, sending a full tablet report...
I20240502 14:01:06.395763 8529 tablet_bootstrap.cc:492] T b3bf9e5109e64e36b7555c6fdd21c504 P 9d58e642482142f7b4e27ce750595df7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:06.396056 8529 tablet_bootstrap.cc:492] T b3bf9e5109e64e36b7555c6fdd21c504 P 9d58e642482142f7b4e27ce750595df7: Bootstrap complete.
I20240502 14:01:06.396293 8529 ts_tablet_manager.cc:1367] T b3bf9e5109e64e36b7555c6fdd21c504 P 9d58e642482142f7b4e27ce750595df7: Time spent bootstrapping tablet: real 0.027s user 0.019s sys 0.004s
I20240502 14:01:06.398258 8529 raft_consensus.cc:348] T b3bf9e5109e64e36b7555c6fdd21c504 P 9d58e642482142f7b4e27ce750595df7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: true } }
I20240502 14:01:06.398465 8529 raft_consensus.cc:729] T b3bf9e5109e64e36b7555c6fdd21c504 P 9d58e642482142f7b4e27ce750595df7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9d58e642482142f7b4e27ce750595df7, State: Initialized, Role: LEARNER
I20240502 14:01:06.398758 8529 consensus_queue.cc:260] T b3bf9e5109e64e36b7555c6fdd21c504 P 9d58e642482142f7b4e27ce750595df7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: true } }
I20240502 14:01:06.399525 8529 ts_tablet_manager.cc:1398] T b3bf9e5109e64e36b7555c6fdd21c504 P 9d58e642482142f7b4e27ce750595df7: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20240502 14:01:06.400086 7817 tablet_copy_service.cc:342] P de464104b5994dcd8e43e854fb53c955: Request end of tablet copy session 9d58e642482142f7b4e27ce750595df7-b3bf9e5109e64e36b7555c6fdd21c504 received from {username='slave'} at 127.24.66.67:48763
I20240502 14:01:06.400168 7817 tablet_copy_service.cc:433] P de464104b5994dcd8e43e854fb53c955: ending tablet copy session 9d58e642482142f7b4e27ce750595df7-b3bf9e5109e64e36b7555c6fdd21c504 on tablet b3bf9e5109e64e36b7555c6fdd21c504 with peer 9d58e642482142f7b4e27ce750595df7
I20240502 14:01:06.418396 8529 ts_tablet_manager.cc:902] T 246de49c933c448b814f04b6085e1166 P 9d58e642482142f7b4e27ce750595df7: Initiating tablet copy from peer de464104b5994dcd8e43e854fb53c955 (127.24.66.68:43765)
I20240502 14:01:06.418888 8529 tablet_copy_client.cc:308] T 246de49c933c448b814f04b6085e1166 P 9d58e642482142f7b4e27ce750595df7: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:43765
I20240502 14:01:06.419342 7817 tablet_copy_service.cc:140] P de464104b5994dcd8e43e854fb53c955: Received BeginTabletCopySession request for tablet 246de49c933c448b814f04b6085e1166 from peer 9d58e642482142f7b4e27ce750595df7 ({username='slave'} at 127.24.66.67:48763)
I20240502 14:01:06.419446 7817 tablet_copy_service.cc:161] P de464104b5994dcd8e43e854fb53c955: Beginning new tablet copy session on tablet 246de49c933c448b814f04b6085e1166 from peer 9d58e642482142f7b4e27ce750595df7 at {username='slave'} at 127.24.66.67:48763: session id = 9d58e642482142f7b4e27ce750595df7-246de49c933c448b814f04b6085e1166
I20240502 14:01:06.420171 7817 tablet_copy_source_session.cc:205] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:06.420991 8529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 246de49c933c448b814f04b6085e1166. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:06.422634 8529 tablet_copy_client.cc:783] T 246de49c933c448b814f04b6085e1166 P 9d58e642482142f7b4e27ce750595df7: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:06.422771 8529 tablet_copy_client.cc:647] T 246de49c933c448b814f04b6085e1166 P 9d58e642482142f7b4e27ce750595df7: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:06.424011 8529 tablet_copy_client.cc:523] T 246de49c933c448b814f04b6085e1166 P 9d58e642482142f7b4e27ce750595df7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:06.425037 8529 tablet_bootstrap.cc:492] T 246de49c933c448b814f04b6085e1166 P 9d58e642482142f7b4e27ce750595df7: Bootstrap starting.
I20240502 14:01:06.440932 8529 tablet_bootstrap.cc:492] T 246de49c933c448b814f04b6085e1166 P 9d58e642482142f7b4e27ce750595df7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:06.441231 8529 tablet_bootstrap.cc:492] T 246de49c933c448b814f04b6085e1166 P 9d58e642482142f7b4e27ce750595df7: Bootstrap complete.
I20240502 14:01:06.441447 8529 ts_tablet_manager.cc:1367] T 246de49c933c448b814f04b6085e1166 P 9d58e642482142f7b4e27ce750595df7: Time spent bootstrapping tablet: real 0.016s user 0.016s sys 0.000s
I20240502 14:01:06.442026 8529 raft_consensus.cc:348] T 246de49c933c448b814f04b6085e1166 P 9d58e642482142f7b4e27ce750595df7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: true } }
I20240502 14:01:06.442137 8529 raft_consensus.cc:729] T 246de49c933c448b814f04b6085e1166 P 9d58e642482142f7b4e27ce750595df7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9d58e642482142f7b4e27ce750595df7, State: Initialized, Role: LEARNER
I20240502 14:01:06.442236 8529 consensus_queue.cc:260] T 246de49c933c448b814f04b6085e1166 P 9d58e642482142f7b4e27ce750595df7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: true } }
I20240502 14:01:06.442494 8529 ts_tablet_manager.cc:1398] T 246de49c933c448b814f04b6085e1166 P 9d58e642482142f7b4e27ce750595df7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:06.443132 7817 tablet_copy_service.cc:342] P de464104b5994dcd8e43e854fb53c955: Request end of tablet copy session 9d58e642482142f7b4e27ce750595df7-246de49c933c448b814f04b6085e1166 received from {username='slave'} at 127.24.66.67:48763
I20240502 14:01:06.443220 7817 tablet_copy_service.cc:433] P de464104b5994dcd8e43e854fb53c955: ending tablet copy session 9d58e642482142f7b4e27ce750595df7-246de49c933c448b814f04b6085e1166 on tablet 246de49c933c448b814f04b6085e1166 with peer 9d58e642482142f7b4e27ce750595df7
I20240502 14:01:06.463876 8515 ts_tablet_manager.cc:902] T c1ef50f64cb94c8bb26dc3c82ebede14 P bf1da42196604fc8a3a2732436afd214: Initiating tablet copy from peer ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65:40249)
I20240502 14:01:06.464390 8515 tablet_copy_client.cc:308] T c1ef50f64cb94c8bb26dc3c82ebede14 P bf1da42196604fc8a3a2732436afd214: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.65:40249
I20240502 14:01:06.464910 7423 tablet_copy_service.cc:140] P ccb6bf69ed6747ad81147d1566054be5: Received BeginTabletCopySession request for tablet c1ef50f64cb94c8bb26dc3c82ebede14 from peer bf1da42196604fc8a3a2732436afd214 ({username='slave'} at 127.24.66.66:57333)
I20240502 14:01:06.465024 7423 tablet_copy_service.cc:161] P ccb6bf69ed6747ad81147d1566054be5: Beginning new tablet copy session on tablet c1ef50f64cb94c8bb26dc3c82ebede14 from peer bf1da42196604fc8a3a2732436afd214 at {username='slave'} at 127.24.66.66:57333: session id = bf1da42196604fc8a3a2732436afd214-c1ef50f64cb94c8bb26dc3c82ebede14
I20240502 14:01:06.465728 7423 tablet_copy_source_session.cc:205] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:06.466531 8515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1ef50f64cb94c8bb26dc3c82ebede14. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:06.468166 8515 tablet_copy_client.cc:783] T c1ef50f64cb94c8bb26dc3c82ebede14 P bf1da42196604fc8a3a2732436afd214: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:06.468307 8515 tablet_copy_client.cc:647] T c1ef50f64cb94c8bb26dc3c82ebede14 P bf1da42196604fc8a3a2732436afd214: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:06.469390 8515 tablet_copy_client.cc:523] T c1ef50f64cb94c8bb26dc3c82ebede14 P bf1da42196604fc8a3a2732436afd214: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:06.470356 8515 tablet_bootstrap.cc:492] T c1ef50f64cb94c8bb26dc3c82ebede14 P bf1da42196604fc8a3a2732436afd214: Bootstrap starting.
I20240502 14:01:06.484784 8529 ts_tablet_manager.cc:902] T ba50f31a27794ff3914210f682e2c421 P 9d58e642482142f7b4e27ce750595df7: Initiating tablet copy from peer de464104b5994dcd8e43e854fb53c955 (127.24.66.68:43765)
I20240502 14:01:06.485267 8529 tablet_copy_client.cc:308] T ba50f31a27794ff3914210f682e2c421 P 9d58e642482142f7b4e27ce750595df7: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:43765
I20240502 14:01:06.485775 7817 tablet_copy_service.cc:140] P de464104b5994dcd8e43e854fb53c955: Received BeginTabletCopySession request for tablet ba50f31a27794ff3914210f682e2c421 from peer 9d58e642482142f7b4e27ce750595df7 ({username='slave'} at 127.24.66.67:48763)
I20240502 14:01:06.485872 7817 tablet_copy_service.cc:161] P de464104b5994dcd8e43e854fb53c955: Beginning new tablet copy session on tablet ba50f31a27794ff3914210f682e2c421 from peer 9d58e642482142f7b4e27ce750595df7 at {username='slave'} at 127.24.66.67:48763: session id = 9d58e642482142f7b4e27ce750595df7-ba50f31a27794ff3914210f682e2c421
I20240502 14:01:06.486614 7817 tablet_copy_source_session.cc:205] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:06.487423 8529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba50f31a27794ff3914210f682e2c421. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:06.489176 8515 tablet_bootstrap.cc:492] T c1ef50f64cb94c8bb26dc3c82ebede14 P bf1da42196604fc8a3a2732436afd214: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:06.489466 8515 tablet_bootstrap.cc:492] T c1ef50f64cb94c8bb26dc3c82ebede14 P bf1da42196604fc8a3a2732436afd214: Bootstrap complete.
I20240502 14:01:06.489650 8515 ts_tablet_manager.cc:1367] T c1ef50f64cb94c8bb26dc3c82ebede14 P bf1da42196604fc8a3a2732436afd214: Time spent bootstrapping tablet: real 0.019s user 0.016s sys 0.003s
I20240502 14:01:06.489660 8529 tablet_copy_client.cc:783] T ba50f31a27794ff3914210f682e2c421 P 9d58e642482142f7b4e27ce750595df7: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:06.489902 8529 tablet_copy_client.cc:647] T ba50f31a27794ff3914210f682e2c421 P 9d58e642482142f7b4e27ce750595df7: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:06.490132 8515 raft_consensus.cc:348] T c1ef50f64cb94c8bb26dc3c82ebede14 P bf1da42196604fc8a3a2732436afd214 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: true } }
I20240502 14:01:06.490229 8515 raft_consensus.cc:729] T c1ef50f64cb94c8bb26dc3c82ebede14 P bf1da42196604fc8a3a2732436afd214 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bf1da42196604fc8a3a2732436afd214, State: Initialized, Role: LEARNER
I20240502 14:01:06.490329 8515 consensus_queue.cc:260] T c1ef50f64cb94c8bb26dc3c82ebede14 P bf1da42196604fc8a3a2732436afd214 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: NON_VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: true } }
I20240502 14:01:06.490633 8515 ts_tablet_manager.cc:1398] T c1ef50f64cb94c8bb26dc3c82ebede14 P bf1da42196604fc8a3a2732436afd214: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:01:06.491039 8529 tablet_copy_client.cc:523] T ba50f31a27794ff3914210f682e2c421 P 9d58e642482142f7b4e27ce750595df7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:06.491135 7423 tablet_copy_service.cc:342] P ccb6bf69ed6747ad81147d1566054be5: Request end of tablet copy session bf1da42196604fc8a3a2732436afd214-c1ef50f64cb94c8bb26dc3c82ebede14 received from {username='slave'} at 127.24.66.66:57333
I20240502 14:01:06.491214 7423 tablet_copy_service.cc:433] P ccb6bf69ed6747ad81147d1566054be5: ending tablet copy session bf1da42196604fc8a3a2732436afd214-c1ef50f64cb94c8bb26dc3c82ebede14 on tablet c1ef50f64cb94c8bb26dc3c82ebede14 with peer bf1da42196604fc8a3a2732436afd214
I20240502 14:01:06.492151 8529 tablet_bootstrap.cc:492] T ba50f31a27794ff3914210f682e2c421 P 9d58e642482142f7b4e27ce750595df7: Bootstrap starting.
I20240502 14:01:06.509124 8529 tablet_bootstrap.cc:492] T ba50f31a27794ff3914210f682e2c421 P 9d58e642482142f7b4e27ce750595df7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:06.509370 8529 tablet_bootstrap.cc:492] T ba50f31a27794ff3914210f682e2c421 P 9d58e642482142f7b4e27ce750595df7: Bootstrap complete.
I20240502 14:01:06.509518 8529 ts_tablet_manager.cc:1367] T ba50f31a27794ff3914210f682e2c421 P 9d58e642482142f7b4e27ce750595df7: Time spent bootstrapping tablet: real 0.017s user 0.015s sys 0.001s
I20240502 14:01:06.509896 8529 raft_consensus.cc:348] T ba50f31a27794ff3914210f682e2c421 P 9d58e642482142f7b4e27ce750595df7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: true } }
I20240502 14:01:06.509990 8529 raft_consensus.cc:729] T ba50f31a27794ff3914210f682e2c421 P 9d58e642482142f7b4e27ce750595df7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9d58e642482142f7b4e27ce750595df7, State: Initialized, Role: LEARNER
I20240502 14:01:06.510083 8529 consensus_queue.cc:260] T ba50f31a27794ff3914210f682e2c421 P 9d58e642482142f7b4e27ce750595df7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: true } }
I20240502 14:01:06.510353 8529 ts_tablet_manager.cc:1398] T ba50f31a27794ff3914210f682e2c421 P 9d58e642482142f7b4e27ce750595df7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:06.510875 7817 tablet_copy_service.cc:342] P de464104b5994dcd8e43e854fb53c955: Request end of tablet copy session 9d58e642482142f7b4e27ce750595df7-ba50f31a27794ff3914210f682e2c421 received from {username='slave'} at 127.24.66.67:48763
I20240502 14:01:06.510953 7817 tablet_copy_service.cc:433] P de464104b5994dcd8e43e854fb53c955: ending tablet copy session 9d58e642482142f7b4e27ce750595df7-ba50f31a27794ff3914210f682e2c421 on tablet ba50f31a27794ff3914210f682e2c421 with peer 9d58e642482142f7b4e27ce750595df7
I20240502 14:01:06.849036 8221 raft_consensus.cc:1203] T 246de49c933c448b814f04b6085e1166 P 9d58e642482142f7b4e27ce750595df7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:01:06.895403 8090 raft_consensus.cc:1203] T 2bb96d00ac544094b41b7f594dda5367 P bf1da42196604fc8a3a2732436afd214 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:01:06.916169 8090 raft_consensus.cc:1203] T 6075e58cdba74c919dbb563ff14389ec P bf1da42196604fc8a3a2732436afd214 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:01:06.922086 8221 raft_consensus.cc:1203] T ba50f31a27794ff3914210f682e2c421 P 9d58e642482142f7b4e27ce750595df7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:01:06.931983 8221 raft_consensus.cc:1203] T b3bf9e5109e64e36b7555c6fdd21c504 P 9d58e642482142f7b4e27ce750595df7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:01:06.961477 8090 raft_consensus.cc:1203] T c1ef50f64cb94c8bb26dc3c82ebede14 P bf1da42196604fc8a3a2732436afd214 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 14:01:07.303189 8640 raft_consensus.cc:1050] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5: attempting to promote NON_VOTER bf1da42196604fc8a3a2732436afd214 to VOTER
I20240502 14:01:07.303597 8640 consensus_queue.cc:237] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } }
I20240502 14:01:07.304678 7800 raft_consensus.cc:1261] T 6075e58cdba74c919dbb563ff14389ec P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Refusing update from remote peer ccb6bf69ed6747ad81147d1566054be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:07.304798 8090 raft_consensus.cc:1261] T 6075e58cdba74c919dbb563ff14389ec P bf1da42196604fc8a3a2732436afd214 [term 1 LEARNER]: Refusing update from remote peer ccb6bf69ed6747ad81147d1566054be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:07.305182 7878 consensus_queue.cc:1035] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:07.305372 8641 consensus_queue.cc:1035] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:07.306996 8641 raft_consensus.cc:2932] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, bf1da42196604fc8a3a2732436afd214 (127.24.66.66) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } } }
I20240502 14:01:07.307422 7800 raft_consensus.cc:2932] T 6075e58cdba74c919dbb563ff14389ec P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, bf1da42196604fc8a3a2732436afd214 (127.24.66.66) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } } }
I20240502 14:01:07.313400 7257 catalog_manager.cc:5484] T 6075e58cdba74c919dbb563ff14389ec P de464104b5994dcd8e43e854fb53c955 reported cstate change: config changed from index 4 to 5, bf1da42196604fc8a3a2732436afd214 (127.24.66.66) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ccb6bf69ed6747ad81147d1566054be5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } } }
I20240502 14:01:07.313421 8090 raft_consensus.cc:2932] T 6075e58cdba74c919dbb563ff14389ec P bf1da42196604fc8a3a2732436afd214 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, bf1da42196604fc8a3a2732436afd214 (127.24.66.66) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } } }
I20240502 14:01:07.318192 7404 consensus_queue.cc:237] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } }
I20240502 14:01:07.318991 8090 raft_consensus.cc:1261] T 6075e58cdba74c919dbb563ff14389ec P bf1da42196604fc8a3a2732436afd214 [term 1 FOLLOWER]: Refusing update from remote peer ccb6bf69ed6747ad81147d1566054be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:07.319288 8641 consensus_queue.cc:1035] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:07.320505 7878 raft_consensus.cc:2932] T 6075e58cdba74c919dbb563ff14389ec P ccb6bf69ed6747ad81147d1566054be5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER de464104b5994dcd8e43e854fb53c955 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } } }
I20240502 14:01:07.320914 8090 raft_consensus.cc:2932] T 6075e58cdba74c919dbb563ff14389ec P bf1da42196604fc8a3a2732436afd214 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER de464104b5994dcd8e43e854fb53c955 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } } }
I20240502 14:01:07.321785 7245 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6075e58cdba74c919dbb563ff14389ec with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:07.322811 7253 catalog_manager.cc:5484] T 6075e58cdba74c919dbb563ff14389ec P bf1da42196604fc8a3a2732436afd214 reported cstate change: config changed from index 5 to 6, VOTER de464104b5994dcd8e43e854fb53c955 (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "ccb6bf69ed6747ad81147d1566054be5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } } }
I20240502 14:01:07.324816 7778 tablet_service.cc:1514] Processing DeleteTablet for tablet 6075e58cdba74c919dbb563ff14389ec with delete_type TABLET_DATA_TOMBSTONED (TS de464104b5994dcd8e43e854fb53c955 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:42388
I20240502 14:01:07.324904 7878 raft_consensus.cc:1050] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5: attempting to promote NON_VOTER bf1da42196604fc8a3a2732436afd214 to VOTER
I20240502 14:01:07.325239 8710 tablet_replica.cc:330] T 6075e58cdba74c919dbb563ff14389ec P de464104b5994dcd8e43e854fb53c955: stopping tablet replica
I20240502 14:01:07.325204 7878 consensus_queue.cc:237] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } }
I20240502 14:01:07.325515 8710 raft_consensus.cc:2229] T 6075e58cdba74c919dbb563ff14389ec P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:07.325713 8710 raft_consensus.cc:2258] T 6075e58cdba74c919dbb563ff14389ec P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:07.326265 7800 raft_consensus.cc:1261] T 2bb96d00ac544094b41b7f594dda5367 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Refusing update from remote peer ccb6bf69ed6747ad81147d1566054be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:07.326282 8090 raft_consensus.cc:1261] T 2bb96d00ac544094b41b7f594dda5367 P bf1da42196604fc8a3a2732436afd214 [term 1 LEARNER]: Refusing update from remote peer ccb6bf69ed6747ad81147d1566054be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:07.326557 7878 consensus_queue.cc:1035] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:07.326741 8640 consensus_queue.cc:1035] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:07.327028 8710 ts_tablet_manager.cc:1875] T 6075e58cdba74c919dbb563ff14389ec P de464104b5994dcd8e43e854fb53c955: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:07.328162 7878 raft_consensus.cc:2932] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, bf1da42196604fc8a3a2732436afd214 (127.24.66.66) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } } }
I20240502 14:01:07.328646 7800 raft_consensus.cc:2932] T 2bb96d00ac544094b41b7f594dda5367 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, bf1da42196604fc8a3a2732436afd214 (127.24.66.66) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } } }
I20240502 14:01:07.329269 8710 ts_tablet_manager.cc:1888] T 6075e58cdba74c919dbb563ff14389ec P de464104b5994dcd8e43e854fb53c955: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:01:07.329358 8710 log.cc:1196] T 6075e58cdba74c919dbb563ff14389ec P de464104b5994dcd8e43e854fb53c955: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/6075e58cdba74c919dbb563ff14389ec
I20240502 14:01:07.330160 7245 catalog_manager.cc:4830] TS de464104b5994dcd8e43e854fb53c955 (127.24.66.68:43765): tablet 6075e58cdba74c919dbb563ff14389ec (table rebalance_test_table_1 [id=ef12fb2c44cf4dcfb39f8e343278d5c4]) successfully deleted
I20240502 14:01:07.330425 8090 raft_consensus.cc:2932] T 2bb96d00ac544094b41b7f594dda5367 P bf1da42196604fc8a3a2732436afd214 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, bf1da42196604fc8a3a2732436afd214 (127.24.66.66) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } } }
I20240502 14:01:07.330484 7255 catalog_manager.cc:5484] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 reported cstate change: config changed from index 4 to 5, bf1da42196604fc8a3a2732436afd214 (127.24.66.66) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ccb6bf69ed6747ad81147d1566054be5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:07.332650 7404 consensus_queue.cc:237] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } }
I20240502 14:01:07.333397 8090 raft_consensus.cc:1261] T 2bb96d00ac544094b41b7f594dda5367 P bf1da42196604fc8a3a2732436afd214 [term 1 FOLLOWER]: Refusing update from remote peer ccb6bf69ed6747ad81147d1566054be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:07.333709 8641 consensus_queue.cc:1035] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:07.334918 7878 raft_consensus.cc:2932] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER de464104b5994dcd8e43e854fb53c955 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } } }
I20240502 14:01:07.335242 8090 raft_consensus.cc:2932] T 2bb96d00ac544094b41b7f594dda5367 P bf1da42196604fc8a3a2732436afd214 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER de464104b5994dcd8e43e854fb53c955 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } } }
I20240502 14:01:07.336287 7245 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 2bb96d00ac544094b41b7f594dda5367 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:07.336612 7257 catalog_manager.cc:5484] T 2bb96d00ac544094b41b7f594dda5367 P ccb6bf69ed6747ad81147d1566054be5 reported cstate change: config changed from index 5 to 6, VOTER de464104b5994dcd8e43e854fb53c955 (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "ccb6bf69ed6747ad81147d1566054be5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:07.338436 7778 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bb96d00ac544094b41b7f594dda5367 with delete_type TABLET_DATA_TOMBSTONED (TS de464104b5994dcd8e43e854fb53c955 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:42388
I20240502 14:01:07.338622 8710 tablet_replica.cc:330] T 2bb96d00ac544094b41b7f594dda5367 P de464104b5994dcd8e43e854fb53c955: stopping tablet replica
I20240502 14:01:07.338726 8710 raft_consensus.cc:2229] T 2bb96d00ac544094b41b7f594dda5367 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:07.338830 8710 raft_consensus.cc:2258] T 2bb96d00ac544094b41b7f594dda5367 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:07.339304 8710 ts_tablet_manager.cc:1875] T 2bb96d00ac544094b41b7f594dda5367 P de464104b5994dcd8e43e854fb53c955: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:07.340559 8710 ts_tablet_manager.cc:1888] T 2bb96d00ac544094b41b7f594dda5367 P de464104b5994dcd8e43e854fb53c955: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:01:07.340615 8710 log.cc:1196] T 2bb96d00ac544094b41b7f594dda5367 P de464104b5994dcd8e43e854fb53c955: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/2bb96d00ac544094b41b7f594dda5367
I20240502 14:01:07.340961 7245 catalog_manager.cc:4830] TS de464104b5994dcd8e43e854fb53c955 (127.24.66.68:43765): tablet 2bb96d00ac544094b41b7f594dda5367 (table rebalance_test_table_0 [id=4beff913099d4bb29ccf75f6ee161e2d]) successfully deleted
I20240502 14:01:07.421264 8408 raft_consensus.cc:1050] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955: attempting to promote NON_VOTER 9d58e642482142f7b4e27ce750595df7 to VOTER
I20240502 14:01:07.421627 8408 consensus_queue.cc:237] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } }
I20240502 14:01:07.422719 7404 raft_consensus.cc:1261] T b3bf9e5109e64e36b7555c6fdd21c504 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Refusing update from remote peer de464104b5994dcd8e43e854fb53c955: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:07.422757 8221 raft_consensus.cc:1261] T b3bf9e5109e64e36b7555c6fdd21c504 P 9d58e642482142f7b4e27ce750595df7 [term 1 LEARNER]: Refusing update from remote peer de464104b5994dcd8e43e854fb53c955: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:07.423091 8639 consensus_queue.cc:1035] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:07.423269 8642 consensus_queue.cc:1035] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:07.425007 8639 raft_consensus.cc:2932] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 9d58e642482142f7b4e27ce750595df7 (127.24.66.67) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } } }
I20240502 14:01:07.425403 7404 raft_consensus.cc:2932] T b3bf9e5109e64e36b7555c6fdd21c504 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 9d58e642482142f7b4e27ce750595df7 (127.24.66.67) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } } }
I20240502 14:01:07.426103 8221 raft_consensus.cc:2932] T b3bf9e5109e64e36b7555c6fdd21c504 P 9d58e642482142f7b4e27ce750595df7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 9d58e642482142f7b4e27ce750595df7 (127.24.66.67) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } } }
I20240502 14:01:07.427158 7253 catalog_manager.cc:5484] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 reported cstate change: config changed from index 4 to 5, 9d58e642482142f7b4e27ce750595df7 (127.24.66.67) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "de464104b5994dcd8e43e854fb53c955" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:07.429991 7800 consensus_queue.cc:237] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } }
I20240502 14:01:07.430768 8221 raft_consensus.cc:1261] T b3bf9e5109e64e36b7555c6fdd21c504 P 9d58e642482142f7b4e27ce750595df7 [term 1 FOLLOWER]: Refusing update from remote peer de464104b5994dcd8e43e854fb53c955: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:07.431088 8642 consensus_queue.cc:1035] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:07.432376 8639 raft_consensus.cc:2932] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } } }
I20240502 14:01:07.432802 8221 raft_consensus.cc:2932] T b3bf9e5109e64e36b7555c6fdd21c504 P 9d58e642482142f7b4e27ce750595df7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } } }
I20240502 14:01:07.433610 7245 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet b3bf9e5109e64e36b7555c6fdd21c504 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:07.434321 7255 catalog_manager.cc:5484] T b3bf9e5109e64e36b7555c6fdd21c504 P de464104b5994dcd8e43e854fb53c955 reported cstate change: config changed from index 5 to 6, VOTER ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65) evicted. New cstate: current_term: 1 leader_uuid: "de464104b5994dcd8e43e854fb53c955" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:07.436269 7384 tablet_service.cc:1514] Processing DeleteTablet for tablet b3bf9e5109e64e36b7555c6fdd21c504 with delete_type TABLET_DATA_TOMBSTONED (TS ccb6bf69ed6747ad81147d1566054be5 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:49860
I20240502 14:01:07.436751 8717 tablet_replica.cc:330] T b3bf9e5109e64e36b7555c6fdd21c504 P ccb6bf69ed6747ad81147d1566054be5: stopping tablet replica
I20240502 14:01:07.437038 8717 raft_consensus.cc:2229] T b3bf9e5109e64e36b7555c6fdd21c504 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:07.437213 8717 raft_consensus.cc:2258] T b3bf9e5109e64e36b7555c6fdd21c504 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:07.438158 8717 ts_tablet_manager.cc:1875] T b3bf9e5109e64e36b7555c6fdd21c504 P ccb6bf69ed6747ad81147d1566054be5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:07.439761 8717 ts_tablet_manager.cc:1888] T b3bf9e5109e64e36b7555c6fdd21c504 P ccb6bf69ed6747ad81147d1566054be5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:01:07.439831 8717 log.cc:1196] T b3bf9e5109e64e36b7555c6fdd21c504 P ccb6bf69ed6747ad81147d1566054be5: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/wals/b3bf9e5109e64e36b7555c6fdd21c504
I20240502 14:01:07.440307 7245 catalog_manager.cc:4830] TS ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65:40249): tablet b3bf9e5109e64e36b7555c6fdd21c504 (table rebalance_test_table_2 [id=6b7ad224eb254c07b5b15e54556a1d33]) successfully deleted
I20240502 14:01:07.466081 8639 raft_consensus.cc:1050] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955: attempting to promote NON_VOTER 9d58e642482142f7b4e27ce750595df7 to VOTER
I20240502 14:01:07.466336 8639 consensus_queue.cc:237] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } }
I20240502 14:01:07.467268 7404 raft_consensus.cc:1261] T 246de49c933c448b814f04b6085e1166 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Refusing update from remote peer de464104b5994dcd8e43e854fb53c955: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:07.467269 8221 raft_consensus.cc:1261] T 246de49c933c448b814f04b6085e1166 P 9d58e642482142f7b4e27ce750595df7 [term 1 LEARNER]: Refusing update from remote peer de464104b5994dcd8e43e854fb53c955: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:07.467579 8642 consensus_queue.cc:1035] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:07.467751 8639 consensus_queue.cc:1035] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:07.469223 8408 raft_consensus.cc:2932] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 9d58e642482142f7b4e27ce750595df7 (127.24.66.67) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } } }
I20240502 14:01:07.469637 7404 raft_consensus.cc:2932] T 246de49c933c448b814f04b6085e1166 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 9d58e642482142f7b4e27ce750595df7 (127.24.66.67) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } } }
I20240502 14:01:07.469738 8221 raft_consensus.cc:2932] T 246de49c933c448b814f04b6085e1166 P 9d58e642482142f7b4e27ce750595df7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 9d58e642482142f7b4e27ce750595df7 (127.24.66.67) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } } }
I20240502 14:01:07.471203 7257 catalog_manager.cc:5484] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 reported cstate change: config changed from index 4 to 5, 9d58e642482142f7b4e27ce750595df7 (127.24.66.67) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "de464104b5994dcd8e43e854fb53c955" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:07.473436 7800 consensus_queue.cc:237] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } }
I20240502 14:01:07.474213 8221 raft_consensus.cc:1261] T 246de49c933c448b814f04b6085e1166 P 9d58e642482142f7b4e27ce750595df7 [term 1 FOLLOWER]: Refusing update from remote peer de464104b5994dcd8e43e854fb53c955: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:07.474500 8639 consensus_queue.cc:1035] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:07.475612 8408 raft_consensus.cc:2932] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } } }
I20240502 14:01:07.476006 8221 raft_consensus.cc:2932] T 246de49c933c448b814f04b6085e1166 P 9d58e642482142f7b4e27ce750595df7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } } }
I20240502 14:01:07.476975 7245 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 246de49c933c448b814f04b6085e1166 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:07.477494 7255 catalog_manager.cc:5484] T 246de49c933c448b814f04b6085e1166 P de464104b5994dcd8e43e854fb53c955 reported cstate change: config changed from index 5 to 6, VOTER ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65) evicted. New cstate: current_term: 1 leader_uuid: "de464104b5994dcd8e43e854fb53c955" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:07.479498 7384 tablet_service.cc:1514] Processing DeleteTablet for tablet 246de49c933c448b814f04b6085e1166 with delete_type TABLET_DATA_TOMBSTONED (TS ccb6bf69ed6747ad81147d1566054be5 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:49860
I20240502 14:01:07.479686 8717 tablet_replica.cc:330] T 246de49c933c448b814f04b6085e1166 P ccb6bf69ed6747ad81147d1566054be5: stopping tablet replica
I20240502 14:01:07.479800 8717 raft_consensus.cc:2229] T 246de49c933c448b814f04b6085e1166 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:07.479929 8717 raft_consensus.cc:2258] T 246de49c933c448b814f04b6085e1166 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:07.480587 8717 ts_tablet_manager.cc:1875] T 246de49c933c448b814f04b6085e1166 P ccb6bf69ed6747ad81147d1566054be5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:07.481480 7878 raft_consensus.cc:1050] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5: attempting to promote NON_VOTER bf1da42196604fc8a3a2732436afd214 to VOTER
I20240502 14:01:07.481765 7878 consensus_queue.cc:237] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } }
I20240502 14:01:07.482187 8717 ts_tablet_manager.cc:1888] T 246de49c933c448b814f04b6085e1166 P ccb6bf69ed6747ad81147d1566054be5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:01:07.482252 8717 log.cc:1196] T 246de49c933c448b814f04b6085e1166 P ccb6bf69ed6747ad81147d1566054be5: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/wals/246de49c933c448b814f04b6085e1166
I20240502 14:01:07.482865 8090 raft_consensus.cc:1261] T c1ef50f64cb94c8bb26dc3c82ebede14 P bf1da42196604fc8a3a2732436afd214 [term 1 LEARNER]: Refusing update from remote peer ccb6bf69ed6747ad81147d1566054be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:07.482924 7800 raft_consensus.cc:1261] T c1ef50f64cb94c8bb26dc3c82ebede14 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Refusing update from remote peer ccb6bf69ed6747ad81147d1566054be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:07.483078 7245 catalog_manager.cc:4830] TS ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65:40249): tablet 246de49c933c448b814f04b6085e1166 (table rebalance_test_table_0 [id=4beff913099d4bb29ccf75f6ee161e2d]) successfully deleted
I20240502 14:01:07.483215 8641 consensus_queue.cc:1035] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:07.483381 8640 consensus_queue.cc:1035] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:07.485013 7878 raft_consensus.cc:2932] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, bf1da42196604fc8a3a2732436afd214 (127.24.66.66) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } } }
I20240502 14:01:07.485463 8090 raft_consensus.cc:2932] T c1ef50f64cb94c8bb26dc3c82ebede14 P bf1da42196604fc8a3a2732436afd214 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, bf1da42196604fc8a3a2732436afd214 (127.24.66.66) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } } }
I20240502 14:01:07.485550 7800 raft_consensus.cc:2932] T c1ef50f64cb94c8bb26dc3c82ebede14 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, bf1da42196604fc8a3a2732436afd214 (127.24.66.66) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } } }
I20240502 14:01:07.487453 7253 catalog_manager.cc:5484] T c1ef50f64cb94c8bb26dc3c82ebede14 P bf1da42196604fc8a3a2732436afd214 reported cstate change: config changed from index 4 to 5, bf1da42196604fc8a3a2732436afd214 (127.24.66.66) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ccb6bf69ed6747ad81147d1566054be5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } attrs { replace: true } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } } }
I20240502 14:01:07.490003 7404 consensus_queue.cc:237] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } }
I20240502 14:01:07.490906 8090 raft_consensus.cc:1261] T c1ef50f64cb94c8bb26dc3c82ebede14 P bf1da42196604fc8a3a2732436afd214 [term 1 FOLLOWER]: Refusing update from remote peer ccb6bf69ed6747ad81147d1566054be5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:07.491216 8641 consensus_queue.cc:1035] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:07.492723 8640 raft_consensus.cc:2932] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER de464104b5994dcd8e43e854fb53c955 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } } }
I20240502 14:01:07.493131 8090 raft_consensus.cc:2932] T c1ef50f64cb94c8bb26dc3c82ebede14 P bf1da42196604fc8a3a2732436afd214 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER de464104b5994dcd8e43e854fb53c955 (127.24.66.68) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } } }
I20240502 14:01:07.494118 7245 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet c1ef50f64cb94c8bb26dc3c82ebede14 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:07.495180 7257 catalog_manager.cc:5484] T c1ef50f64cb94c8bb26dc3c82ebede14 P ccb6bf69ed6747ad81147d1566054be5 reported cstate change: config changed from index 5 to 6, VOTER de464104b5994dcd8e43e854fb53c955 (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "ccb6bf69ed6747ad81147d1566054be5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf1da42196604fc8a3a2732436afd214" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38153 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:07.497505 7778 tablet_service.cc:1514] Processing DeleteTablet for tablet c1ef50f64cb94c8bb26dc3c82ebede14 with delete_type TABLET_DATA_TOMBSTONED (TS de464104b5994dcd8e43e854fb53c955 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:42388
I20240502 14:01:07.497742 8710 tablet_replica.cc:330] T c1ef50f64cb94c8bb26dc3c82ebede14 P de464104b5994dcd8e43e854fb53c955: stopping tablet replica
I20240502 14:01:07.497880 8710 raft_consensus.cc:2229] T c1ef50f64cb94c8bb26dc3c82ebede14 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:07.498018 8710 raft_consensus.cc:2258] T c1ef50f64cb94c8bb26dc3c82ebede14 P de464104b5994dcd8e43e854fb53c955 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:07.498701 8710 ts_tablet_manager.cc:1875] T c1ef50f64cb94c8bb26dc3c82ebede14 P de464104b5994dcd8e43e854fb53c955: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:07.499919 8639 raft_consensus.cc:1050] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955: attempting to promote NON_VOTER 9d58e642482142f7b4e27ce750595df7 to VOTER
I20240502 14:01:07.500226 8639 consensus_queue.cc:237] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } }
I20240502 14:01:07.500617 8710 ts_tablet_manager.cc:1888] T c1ef50f64cb94c8bb26dc3c82ebede14 P de464104b5994dcd8e43e854fb53c955: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:01:07.500679 8710 log.cc:1196] T c1ef50f64cb94c8bb26dc3c82ebede14 P de464104b5994dcd8e43e854fb53c955: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/c1ef50f64cb94c8bb26dc3c82ebede14
I20240502 14:01:07.501204 7245 catalog_manager.cc:4830] TS de464104b5994dcd8e43e854fb53c955 (127.24.66.68:43765): tablet c1ef50f64cb94c8bb26dc3c82ebede14 (table rebalance_test_table_2 [id=6b7ad224eb254c07b5b15e54556a1d33]) successfully deleted
I20240502 14:01:07.501457 7404 raft_consensus.cc:1261] T ba50f31a27794ff3914210f682e2c421 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Refusing update from remote peer de464104b5994dcd8e43e854fb53c955: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:07.501596 8221 raft_consensus.cc:1261] T ba50f31a27794ff3914210f682e2c421 P 9d58e642482142f7b4e27ce750595df7 [term 1 LEARNER]: Refusing update from remote peer de464104b5994dcd8e43e854fb53c955: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:07.501796 8408 consensus_queue.cc:1035] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:07.501998 8639 consensus_queue.cc:1035] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:07.503798 8642 raft_consensus.cc:2932] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 9d58e642482142f7b4e27ce750595df7 (127.24.66.67) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } } }
I20240502 14:01:07.504747 8221 raft_consensus.cc:2932] T ba50f31a27794ff3914210f682e2c421 P 9d58e642482142f7b4e27ce750595df7 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 9d58e642482142f7b4e27ce750595df7 (127.24.66.67) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } } }
I20240502 14:01:07.506330 7255 catalog_manager.cc:5484] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 reported cstate change: config changed from index 4 to 5, 9d58e642482142f7b4e27ce750595df7 (127.24.66.67) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "de464104b5994dcd8e43e854fb53c955" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:07.506336 7404 raft_consensus.cc:2932] T ba50f31a27794ff3914210f682e2c421 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, 9d58e642482142f7b4e27ce750595df7 (127.24.66.67) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "ccb6bf69ed6747ad81147d1566054be5" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 40249 } attrs { replace: true } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } } }
I20240502 14:01:07.509999 7800 consensus_queue.cc:237] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } }
I20240502 14:01:07.511525 8221 raft_consensus.cc:1261] T ba50f31a27794ff3914210f682e2c421 P 9d58e642482142f7b4e27ce750595df7 [term 1 FOLLOWER]: Refusing update from remote peer de464104b5994dcd8e43e854fb53c955: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:07.511951 8639 consensus_queue.cc:1035] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:07.513537 8642 raft_consensus.cc:2932] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } } }
I20240502 14:01:07.513983 8221 raft_consensus.cc:2932] T ba50f31a27794ff3914210f682e2c421 P 9d58e642482142f7b4e27ce750595df7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } } }
I20240502 14:01:07.515383 7245 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ba50f31a27794ff3914210f682e2c421 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:07.515894 7255 catalog_manager.cc:5484] T ba50f31a27794ff3914210f682e2c421 P de464104b5994dcd8e43e854fb53c955 reported cstate change: config changed from index 5 to 6, VOTER ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65) evicted. New cstate: current_term: 1 leader_uuid: "de464104b5994dcd8e43e854fb53c955" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "de464104b5994dcd8e43e854fb53c955" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 43765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d58e642482142f7b4e27ce750595df7" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 36913 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:07.518435 7384 tablet_service.cc:1514] Processing DeleteTablet for tablet ba50f31a27794ff3914210f682e2c421 with delete_type TABLET_DATA_TOMBSTONED (TS ccb6bf69ed6747ad81147d1566054be5 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:49860
I20240502 14:01:07.518673 8717 tablet_replica.cc:330] T ba50f31a27794ff3914210f682e2c421 P ccb6bf69ed6747ad81147d1566054be5: stopping tablet replica
I20240502 14:01:07.518832 8717 raft_consensus.cc:2229] T ba50f31a27794ff3914210f682e2c421 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:07.518990 8717 raft_consensus.cc:2258] T ba50f31a27794ff3914210f682e2c421 P ccb6bf69ed6747ad81147d1566054be5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:07.519587 8717 ts_tablet_manager.cc:1875] T ba50f31a27794ff3914210f682e2c421 P ccb6bf69ed6747ad81147d1566054be5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:07.521533 8717 ts_tablet_manager.cc:1888] T ba50f31a27794ff3914210f682e2c421 P ccb6bf69ed6747ad81147d1566054be5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.5
I20240502 14:01:07.521606 8717 log.cc:1196] T ba50f31a27794ff3914210f682e2c421 P ccb6bf69ed6747ad81147d1566054be5: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.Rebalance_idx7_rf2_t3_l_1_2_1.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/wals/ba50f31a27794ff3914210f682e2c421
I20240502 14:01:07.522105 7245 catalog_manager.cc:4830] TS ccb6bf69ed6747ad81147d1566054be5 (127.24.66.65:40249): tablet ba50f31a27794ff3914210f682e2c421 (table rebalance_test_table_1 [id=ef12fb2c44cf4dcfb39f8e343278d5c4]) successfully deleted
I20240502 14:01:07.596227 7384 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:01:07.596496 8070 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:07.597558 8201 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:07.598397 7778 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:01:07.684815 8201 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:07.684988 8070 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:07.693277 7384 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:01:07.693488 7778 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:01:07.764904 8070 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:07.765314 7778 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:01:07.767037 8201 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:07.774791 7384 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:01:07.808831 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 7321
I20240502 14:01:07.815665 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 8008
I20240502 14:01:07.821897 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 8139
I20240502 14:01:07.827873 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 7717
I20240502 14:01:07.835033 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 7230
2024-05-02T14:01:07Z chronyd exiting
[ OK ] LocationAwareRebalancingParamTest.Rebalance/idx7_rf2_t3_l_1_2_1 (8956 ms)
[ RUN ] LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer/idx6_rf7_t7_l_3_4_3
I20240502 14:01:07.860824 24841 test_util.cc:268] Using random seed: 149115219
I20240502 14:01:07.861652 24841 ts_itest-base.cc:112] Starting cluster with:
I20240502 14:01:07.861693 24841 ts_itest-base.cc:113] --------------
I20240502 14:01:07.861704 24841 ts_itest-base.cc:114] 10 tablet servers
I20240502 14:01:07.861714 24841 ts_itest-base.cc:115] 7 replicas per TS
I20240502 14:01:07.861724 24841 ts_itest-base.cc:116] --------------
2024-05-02T14:01:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T14:01:07Z Disabled control of system clock
I20240502 14:01:07.870007 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:40003
--webserver_interface=127.24.66.126
--webserver_port=0
--builtin_ntp_servers=127.24.66.84:34375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.66.126:40003
--location_mapping_cmd=/tmp/dist-test-taskfX8GOh/build/debug/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/location-assignment.state --map /A:3 --map /B:4 --map /C:3
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=true
--enable_range_replica_placement=false
--master_client_location_assignment_enabled=false
--allow_unsafe_replication_factor with env {}
W20240502 14:01:07.975446 8865 flags.cc:425] Enabled unsafe flag: --allow_unsafe_replication_factor=true
W20240502 14:01:07.975689 8865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:07.975720 8865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:07.975764 8865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:07.978811 8865 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:07.978880 8865 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 14:01:07.978911 8865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:07.978935 8865 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 14:01:07.978951 8865 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 14:01:07.982477 8865 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
--allow_unsafe_replication_factor=true
--enable_range_replica_placement=false
--location_mapping_cmd=/tmp/dist-test-taskfX8GOh/build/debug/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/location-assignment.state --map /A:3 --map /B:4 --map /C:3
--ipki_ca_key_size=768
--master_addresses=127.24.66.126:40003
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.66.126:40003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.24.66.126
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.8865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:07.983283 8865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:07.984315 8865 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:07.992048 8871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:07.992676 8873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:07.993304 8870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:07.994033 8865 server_base.cc:1034] running on GCE node
I20240502 14:01:07.994529 8865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:07.995426 8865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:07.996582 8865 hybrid_clock.cc:648] HybridClock initialized: now 1714658467996567 us; error 24 us; skew 500 ppm
I20240502 14:01:07.998488 8865 webserver.cc:460] Webserver started at http://127.24.66.126:45169/ using document root <none> and password file <none>
I20240502 14:01:07.999058 8865 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:07.999122 8865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:07.999293 8865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:01:08.001000 8865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "a27d956ca0b848cf98605d5412ba045d"
format_stamp: "Formatted at 2024-05-02 14:01:07 on dist-test-slave-5npg"
I20240502 14:01:08.001478 8865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "a27d956ca0b848cf98605d5412ba045d"
format_stamp: "Formatted at 2024-05-02 14:01:07 on dist-test-slave-5npg"
I20240502 14:01:08.004813 8865 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 14:01:08.006973 8879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:08.007972 8865 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:08.008102 8865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
uuid: "a27d956ca0b848cf98605d5412ba045d"
format_stamp: "Formatted at 2024-05-02 14:01:07 on dist-test-slave-5npg"
I20240502 14:01:08.008188 8865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:08.031809 8865 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:08.032507 8865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:08.032673 8865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:08.038972 8931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.126:40003 every 8 connection(s)
I20240502 14:01:08.038977 8865 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.126:40003
I20240502 14:01:08.039988 8865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 14:01:08.042515 8932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:08.045079 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 8865
I20240502 14:01:08.045226 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 14:01:08.048067 8932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a27d956ca0b848cf98605d5412ba045d: Bootstrap starting.
I20240502 14:01:08.050470 8932 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a27d956ca0b848cf98605d5412ba045d: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:08.051326 8932 log.cc:824] T 00000000000000000000000000000000 P a27d956ca0b848cf98605d5412ba045d: Log is configured to *not* fsync() on all Append() calls
I20240502 14:01:08.053081 8932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a27d956ca0b848cf98605d5412ba045d: No bootstrap required, opened a new log
I20240502 14:01:08.055636 8932 raft_consensus.cc:348] T 00000000000000000000000000000000 P a27d956ca0b848cf98605d5412ba045d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a27d956ca0b848cf98605d5412ba045d" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40003 } }
I20240502 14:01:08.055824 8932 raft_consensus.cc:374] T 00000000000000000000000000000000 P a27d956ca0b848cf98605d5412ba045d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:08.055889 8932 raft_consensus.cc:729] T 00000000000000000000000000000000 P a27d956ca0b848cf98605d5412ba045d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a27d956ca0b848cf98605d5412ba045d, State: Initialized, Role: FOLLOWER
I20240502 14:01:08.056279 8932 consensus_queue.cc:260] T 00000000000000000000000000000000 P a27d956ca0b848cf98605d5412ba045d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a27d956ca0b848cf98605d5412ba045d" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40003 } }
I20240502 14:01:08.056437 8932 raft_consensus.cc:388] T 00000000000000000000000000000000 P a27d956ca0b848cf98605d5412ba045d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 14:01:08.056499 8932 raft_consensus.cc:482] T 00000000000000000000000000000000 P a27d956ca0b848cf98605d5412ba045d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 14:01:08.056614 8932 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a27d956ca0b848cf98605d5412ba045d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:08.057369 8932 raft_consensus.cc:504] T 00000000000000000000000000000000 P a27d956ca0b848cf98605d5412ba045d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a27d956ca0b848cf98605d5412ba045d" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40003 } }
I20240502 14:01:08.057770 8932 leader_election.cc:302] T 00000000000000000000000000000000 P a27d956ca0b848cf98605d5412ba045d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a27d956ca0b848cf98605d5412ba045d; no voters:
I20240502 14:01:08.058066 8932 leader_election.cc:288] T 00000000000000000000000000000000 P a27d956ca0b848cf98605d5412ba045d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 14:01:08.058154 8937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a27d956ca0b848cf98605d5412ba045d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:08.058333 8937 raft_consensus.cc:686] T 00000000000000000000000000000000 P a27d956ca0b848cf98605d5412ba045d [term 1 LEADER]: Becoming Leader. State: Replica: a27d956ca0b848cf98605d5412ba045d, State: Running, Role: LEADER
I20240502 14:01:08.058616 8937 consensus_queue.cc:237] T 00000000000000000000000000000000 P a27d956ca0b848cf98605d5412ba045d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a27d956ca0b848cf98605d5412ba045d" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40003 } }
I20240502 14:01:08.059180 8932 sys_catalog.cc:564] T 00000000000000000000000000000000 P a27d956ca0b848cf98605d5412ba045d [sys.catalog]: configured and running, proceeding with master startup.
I20240502 14:01:08.060189 8938 sys_catalog.cc:455] T 00000000000000000000000000000000 P a27d956ca0b848cf98605d5412ba045d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a27d956ca0b848cf98605d5412ba045d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a27d956ca0b848cf98605d5412ba045d" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40003 } } }
I20240502 14:01:08.060360 8938 sys_catalog.cc:458] T 00000000000000000000000000000000 P a27d956ca0b848cf98605d5412ba045d [sys.catalog]: This master's current role is: LEADER
I20240502 14:01:08.060611 8939 sys_catalog.cc:455] T 00000000000000000000000000000000 P a27d956ca0b848cf98605d5412ba045d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a27d956ca0b848cf98605d5412ba045d. Latest consensus state: current_term: 1 leader_uuid: "a27d956ca0b848cf98605d5412ba045d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a27d956ca0b848cf98605d5412ba045d" member_type: VOTER last_known_addr { host: "127.24.66.126" port: 40003 } } }
I20240502 14:01:08.060698 8939 sys_catalog.cc:458] T 00000000000000000000000000000000 P a27d956ca0b848cf98605d5412ba045d [sys.catalog]: This master's current role is: LEADER
I20240502 14:01:08.060743 8946 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 14:01:08.063313 8946 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 14:01:08.068279 8946 catalog_manager.cc:1347] Generated new cluster ID: 6f462d1b3dad4e79af6335cc9a4b77c7
I20240502 14:01:08.068377 8946 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 14:01:08.083082 8946 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 14:01:08.084286 8946 catalog_manager.cc:1504] Loading token signing keys...
I20240502 14:01:08.094971 8946 catalog_manager.cc:5857] T 00000000000000000000000000000000 P a27d956ca0b848cf98605d5412ba045d: Generated new TSK 0
I20240502 14:01:08.095772 8946 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 14:01:08.098217 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.65:0
--local_ip_for_outbound_sockets=127.24.66.65
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40003
--builtin_ntp_servers=127.24.66.84:34375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 14:01:08.206490 8956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:08.206707 8956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:08.206754 8956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:08.209831 8956 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:08.209898 8956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:08.209971 8956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.65
I20240502 14:01:08.213845 8956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.24.66.65
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40003
--never_fsync=true
--heap_profile_path=/tmp/kudu.8956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.65
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:08.214836 8956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:08.216012 8956 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:08.225234 8962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:08.226002 8964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:08.226565 8961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:08.227339 8956 server_base.cc:1034] running on GCE node
I20240502 14:01:08.227741 8956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:08.228359 8956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:08.229517 8956 hybrid_clock.cc:648] HybridClock initialized: now 1714658468229496 us; error 37 us; skew 500 ppm
I20240502 14:01:08.231415 8956 webserver.cc:460] Webserver started at http://127.24.66.65:37665/ using document root <none> and password file <none>
I20240502 14:01:08.231947 8956 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:08.231998 8956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:08.232149 8956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:01:08.233837 8956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d"
format_stamp: "Formatted at 2024-05-02 14:01:08 on dist-test-slave-5npg"
I20240502 14:01:08.234273 8956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d"
format_stamp: "Formatted at 2024-05-02 14:01:08 on dist-test-slave-5npg"
I20240502 14:01:08.237557 8956 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20240502 14:01:08.239658 8970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:08.240660 8956 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240502 14:01:08.240752 8956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d"
format_stamp: "Formatted at 2024-05-02 14:01:08 on dist-test-slave-5npg"
I20240502 14:01:08.240825 8956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:08.251128 8956 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:08.251677 8956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:08.251808 8956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:08.252265 8956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:08.253152 8956 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:08.253198 8956 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:08.253237 8956 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:08.253257 8956 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:08.261529 8956 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.65:45459
I20240502 14:01:08.261580 9083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.65:45459 every 8 connection(s)
I20240502 14:01:08.262470 8956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 14:01:08.263777 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 8956
I20240502 14:01:08.263872 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 14:01:08.266160 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.66:0
--local_ip_for_outbound_sockets=127.24.66.66
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40003
--builtin_ntp_servers=127.24.66.84:34375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:01:08.271651 9084 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40003
I20240502 14:01:08.271869 9084 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:08.272454 9084 heartbeater.cc:511] Master 127.24.66.126:40003 requested a full tablet report, sending...
I20240502 14:01:08.308526 8895 ts_manager.cc:194] Registered new tserver with Master: bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459)
I20240502 14:01:08.310436 8895 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.65:52359
W20240502 14:01:08.381179 9087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:08.381384 9087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:08.381444 9087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:08.384426 9087 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:08.384482 9087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:08.384546 9087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.66
I20240502 14:01:08.387967 9087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.24.66.66
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40003
--never_fsync=true
--heap_profile_path=/tmp/kudu.9087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.66
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:08.388790 9087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:08.389854 9087 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:08.398458 9094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:08.399053 9093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:08.399745 9096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:08.400522 9087 server_base.cc:1034] running on GCE node
I20240502 14:01:08.400885 9087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:08.401417 9087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:08.402583 9087 hybrid_clock.cc:648] HybridClock initialized: now 1714658468402570 us; error 25 us; skew 500 ppm
I20240502 14:01:08.404409 9087 webserver.cc:460] Webserver started at http://127.24.66.66:45669/ using document root <none> and password file <none>
I20240502 14:01:08.404942 9087 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:08.405004 9087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:08.405174 9087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:01:08.406760 9087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "d83af1bc823a4f2ca531f0d6825f8e16"
format_stamp: "Formatted at 2024-05-02 14:01:08 on dist-test-slave-5npg"
I20240502 14:01:08.407217 9087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "d83af1bc823a4f2ca531f0d6825f8e16"
format_stamp: "Formatted at 2024-05-02 14:01:08 on dist-test-slave-5npg"
I20240502 14:01:08.410574 9087 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 14:01:08.412698 9102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:08.413659 9087 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:08.413761 9087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "d83af1bc823a4f2ca531f0d6825f8e16"
format_stamp: "Formatted at 2024-05-02 14:01:08 on dist-test-slave-5npg"
I20240502 14:01:08.413844 9087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:08.441198 9087 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:08.441852 9087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:08.442015 9087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:08.442618 9087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:08.443543 9087 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:08.443595 9087 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:08.443662 9087 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:08.443691 9087 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:08.452296 9087 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.66:38837
I20240502 14:01:08.452353 9215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.66:38837 every 8 connection(s)
I20240502 14:01:08.453336 9087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 14:01:08.462497 9216 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40003
I20240502 14:01:08.462803 9216 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:08.462909 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 9087
I20240502 14:01:08.463042 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 14:01:08.463444 9216 heartbeater.cc:511] Master 127.24.66.126:40003 requested a full tablet report, sending...
I20240502 14:01:08.465102 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.67:0
--local_ip_for_outbound_sockets=127.24.66.67
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40003
--builtin_ntp_servers=127.24.66.84:34375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:01:08.499588 8896 ts_manager.cc:194] Registered new tserver with Master: d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837)
I20240502 14:01:08.500612 8896 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.66:41495
W20240502 14:01:08.572109 9220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:08.572381 9220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:08.572448 9220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:08.575558 9220 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:08.575632 9220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:08.575702 9220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.67
I20240502 14:01:08.579461 9220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.24.66.67
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40003
--never_fsync=true
--heap_profile_path=/tmp/kudu.9220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.67
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:08.580402 9220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:08.581492 9220 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:08.590003 9225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:08.590646 9226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:08.591274 9228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:08.592047 9220 server_base.cc:1034] running on GCE node
I20240502 14:01:08.592492 9220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:08.593056 9220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:08.594200 9220 hybrid_clock.cc:648] HybridClock initialized: now 1714658468594178 us; error 38 us; skew 500 ppm
I20240502 14:01:08.596228 9220 webserver.cc:460] Webserver started at http://127.24.66.67:46165/ using document root <none> and password file <none>
I20240502 14:01:08.596837 9220 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:08.596895 9220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:08.597071 9220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:01:08.598742 9220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "ccfb5bbc1bb54793812d11170ac2d25f"
format_stamp: "Formatted at 2024-05-02 14:01:08 on dist-test-slave-5npg"
I20240502 14:01:08.599239 9220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "ccfb5bbc1bb54793812d11170ac2d25f"
format_stamp: "Formatted at 2024-05-02 14:01:08 on dist-test-slave-5npg"
I20240502 14:01:08.602754 9220 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 14:01:08.604902 9234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:08.605907 9220 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:08.606024 9220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "ccfb5bbc1bb54793812d11170ac2d25f"
format_stamp: "Formatted at 2024-05-02 14:01:08 on dist-test-slave-5npg"
I20240502 14:01:08.606109 9220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:08.631001 9220 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:08.631683 9220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:08.631866 9220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:08.632452 9220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:08.633364 9220 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:08.633414 9220 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:08.633483 9220 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:08.633512 9220 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:08.641495 9220 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.67:34749
I20240502 14:01:08.641518 9347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.67:34749 every 8 connection(s)
I20240502 14:01:08.642390 9220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 14:01:08.650537 9348 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40003
I20240502 14:01:08.650732 9348 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:08.651110 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 9220
I20240502 14:01:08.651223 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 14:01:08.651206 9348 heartbeater.cc:511] Master 127.24.66.126:40003 requested a full tablet report, sending...
I20240502 14:01:08.655181 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.68:0
--local_ip_for_outbound_sockets=127.24.66.68
--webserver_interface=127.24.66.68
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40003
--builtin_ntp_servers=127.24.66.84:34375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:01:08.685464 8896 ts_manager.cc:194] Registered new tserver with Master: ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:08.686414 8896 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.67:50269
W20240502 14:01:08.766006 9352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:08.766222 9352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:08.766265 9352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:08.769322 9352 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:08.769389 9352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:08.769454 9352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.68
I20240502 14:01:08.773190 9352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.68:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.24.66.68
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40003
--never_fsync=true
--heap_profile_path=/tmp/kudu.9352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.68
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:08.774109 9352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:08.775210 9352 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:08.784271 9360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:08.784878 9357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:08.785553 9358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:08.786309 9352 server_base.cc:1034] running on GCE node
I20240502 14:01:08.786738 9352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:08.787322 9352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:08.788470 9352 hybrid_clock.cc:648] HybridClock initialized: now 1714658468788456 us; error 28 us; skew 500 ppm
I20240502 14:01:08.795822 9352 webserver.cc:460] Webserver started at http://127.24.66.68:44189/ using document root <none> and password file <none>
I20240502 14:01:08.796470 9352 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:08.796536 9352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:08.796749 9352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:01:08.798409 9352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "1a490f170c614ee4900ef16c3287c7de"
format_stamp: "Formatted at 2024-05-02 14:01:08 on dist-test-slave-5npg"
I20240502 14:01:08.798897 9352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "1a490f170c614ee4900ef16c3287c7de"
format_stamp: "Formatted at 2024-05-02 14:01:08 on dist-test-slave-5npg"
I20240502 14:01:08.802104 9352 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 14:01:08.804144 9366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:08.805100 9352 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:08.805203 9352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "1a490f170c614ee4900ef16c3287c7de"
format_stamp: "Formatted at 2024-05-02 14:01:08 on dist-test-slave-5npg"
I20240502 14:01:08.805281 9352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:08.817106 9352 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:08.817639 9352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:08.817780 9352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:08.818238 9352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:08.819032 9352 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:08.819079 9352 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:08.819132 9352 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:08.819166 9352 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:08.826942 9352 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.68:42907
I20240502 14:01:08.826982 9479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.68:42907 every 8 connection(s)
I20240502 14:01:08.827754 9352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 14:01:08.833396 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 9352
I20240502 14:01:08.833524 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 14:01:08.835587 9480 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40003
I20240502 14:01:08.835805 9480 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:08.835922 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.69:0
--local_ip_for_outbound_sockets=127.24.66.69
--webserver_interface=127.24.66.69
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40003
--builtin_ntp_servers=127.24.66.84:34375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:01:08.836412 9480 heartbeater.cc:511] Master 127.24.66.126:40003 requested a full tablet report, sending...
I20240502 14:01:08.885376 8895 ts_manager.cc:194] Registered new tserver with Master: 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907)
I20240502 14:01:08.886159 8895 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.68:48569
W20240502 14:01:08.962251 9483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:08.962462 9483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:08.962524 9483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:08.965595 9483 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:08.965654 9483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:08.965720 9483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.69
I20240502 14:01:08.969251 9483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.69:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.24.66.69
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40003
--never_fsync=true
--heap_profile_path=/tmp/kudu.9483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.69
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:08.970095 9483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:08.971171 9483 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:08.979787 9489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:08.980456 9490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:08.981084 9492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:08.981789 9483 server_base.cc:1034] running on GCE node
I20240502 14:01:08.982183 9483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:08.982760 9483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:08.983928 9483 hybrid_clock.cc:648] HybridClock initialized: now 1714658468983917 us; error 32 us; skew 500 ppm
I20240502 14:01:08.991017 9483 webserver.cc:460] Webserver started at http://127.24.66.69:34903/ using document root <none> and password file <none>
I20240502 14:01:08.991605 9483 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:08.991668 9483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:08.991847 9483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:01:08.993593 9483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "d0f45862d7f14ce1976c1a37842f25ca"
format_stamp: "Formatted at 2024-05-02 14:01:08 on dist-test-slave-5npg"
I20240502 14:01:08.994074 9483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "d0f45862d7f14ce1976c1a37842f25ca"
format_stamp: "Formatted at 2024-05-02 14:01:08 on dist-test-slave-5npg"
I20240502 14:01:08.997464 9483 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 14:01:08.999589 9498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:09.000648 9483 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 14:01:09.000757 9483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "d0f45862d7f14ce1976c1a37842f25ca"
format_stamp: "Formatted at 2024-05-02 14:01:08 on dist-test-slave-5npg"
I20240502 14:01:09.000840 9483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:09.013291 9483 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:09.013890 9483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:09.014043 9483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:09.014557 9483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:09.015422 9483 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:09.015470 9483 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:09.015525 9483 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:09.015560 9483 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:09.024122 9483 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.69:36375
I20240502 14:01:09.024140 9611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.69:36375 every 8 connection(s)
I20240502 14:01:09.025077 9483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 14:01:09.032801 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 9483
I20240502 14:01:09.032905 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 14:01:09.033819 9612 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40003
I20240502 14:01:09.034025 9612 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:09.034579 9612 heartbeater.cc:511] Master 127.24.66.126:40003 requested a full tablet report, sending...
I20240502 14:01:09.035127 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.70:0
--local_ip_for_outbound_sockets=127.24.66.70
--webserver_interface=127.24.66.70
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40003
--builtin_ntp_servers=127.24.66.84:34375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:01:09.082451 8895 ts_manager.cc:194] Registered new tserver with Master: d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375)
I20240502 14:01:09.083133 8895 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.69:45671
W20240502 14:01:09.171880 9615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:09.172127 9615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:09.172200 9615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:09.175426 9615 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:09.175514 9615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:09.175630 9615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.70
I20240502 14:01:09.179423 9615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.70:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.24.66.70
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40003
--never_fsync=true
--heap_profile_path=/tmp/kudu.9615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.70
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:09.180385 9615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:09.181514 9615 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:09.190387 9621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:09.191002 9624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:09.191741 9615 server_base.cc:1034] running on GCE node
W20240502 14:01:09.192276 9622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:09.192790 9615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:09.193595 9615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:09.194752 9615 hybrid_clock.cc:648] HybridClock initialized: now 1714658469194740 us; error 27 us; skew 500 ppm
I20240502 14:01:09.196748 9615 webserver.cc:460] Webserver started at http://127.24.66.70:46513/ using document root <none> and password file <none>
I20240502 14:01:09.197350 9615 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:09.197412 9615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:09.197592 9615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:01:09.199286 9615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "efa836edab8d4853a7ca47a23117e288"
format_stamp: "Formatted at 2024-05-02 14:01:09 on dist-test-slave-5npg"
I20240502 14:01:09.199751 9615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "efa836edab8d4853a7ca47a23117e288"
format_stamp: "Formatted at 2024-05-02 14:01:09 on dist-test-slave-5npg"
I20240502 14:01:09.203303 9615 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20240502 14:01:09.205612 9630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:09.206619 9615 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:09.206737 9615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "efa836edab8d4853a7ca47a23117e288"
format_stamp: "Formatted at 2024-05-02 14:01:09 on dist-test-slave-5npg"
I20240502 14:01:09.206825 9615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:09.223166 9615 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:09.223698 9615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:09.223839 9615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:09.224378 9615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:09.225201 9615 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:09.225248 9615 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:09.225302 9615 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:09.225334 9615 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:09.233335 9615 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.70:43397
I20240502 14:01:09.233372 9743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.70:43397 every 8 connection(s)
I20240502 14:01:09.234165 9615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 14:01:09.242102 9744 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40003
I20240502 14:01:09.242300 9744 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:09.242767 9744 heartbeater.cc:511] Master 127.24.66.126:40003 requested a full tablet report, sending...
I20240502 14:01:09.242866 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 9615
I20240502 14:01:09.242986 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240502 14:01:09.246979 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.71:0
--local_ip_for_outbound_sockets=127.24.66.71
--webserver_interface=127.24.66.71
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40003
--builtin_ntp_servers=127.24.66.84:34375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:01:09.277961 8895 ts_manager.cc:194] Registered new tserver with Master: efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397)
I20240502 14:01:09.278707 8895 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.70:36289
I20240502 14:01:09.313071 9084 heartbeater.cc:503] Master 127.24.66.126:40003 was elected leader, sending a full tablet report...
W20240502 14:01:09.358610 9748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:09.358819 9748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:09.358896 9748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:09.362144 9748 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:09.362215 9748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:09.362308 9748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.71
I20240502 14:01:09.365836 9748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.71:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.24.66.71
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40003
--never_fsync=true
--heap_profile_path=/tmp/kudu.9748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.71
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:09.366643 9748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:09.367715 9748 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:09.376473 9754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:09.377072 9753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:09.377771 9756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:09.378505 9748 server_base.cc:1034] running on GCE node
I20240502 14:01:09.378906 9748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:09.379473 9748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:09.380621 9748 hybrid_clock.cc:648] HybridClock initialized: now 1714658469380612 us; error 26 us; skew 500 ppm
I20240502 14:01:09.382611 9748 webserver.cc:460] Webserver started at http://127.24.66.71:44685/ using document root <none> and password file <none>
I20240502 14:01:09.383169 9748 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:09.383231 9748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:09.383419 9748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:01:09.385113 9748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "0488b0e4ed994db19d40aed8c15f945b"
format_stamp: "Formatted at 2024-05-02 14:01:09 on dist-test-slave-5npg"
I20240502 14:01:09.385577 9748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "0488b0e4ed994db19d40aed8c15f945b"
format_stamp: "Formatted at 2024-05-02 14:01:09 on dist-test-slave-5npg"
I20240502 14:01:09.389027 9748 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 14:01:09.391268 9762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:09.392261 9748 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:09.392406 9748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "0488b0e4ed994db19d40aed8c15f945b"
format_stamp: "Formatted at 2024-05-02 14:01:09 on dist-test-slave-5npg"
I20240502 14:01:09.392496 9748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:09.414180 9748 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:09.414785 9748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:09.414943 9748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:09.415457 9748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:09.416370 9748 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:09.416422 9748 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:09.416473 9748 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:09.416508 9748 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:09.424772 9748 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.71:33679
I20240502 14:01:09.424810 9875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.71:33679 every 8 connection(s)
I20240502 14:01:09.425709 9748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 14:01:09.432904 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 9748
I20240502 14:01:09.433058 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240502 14:01:09.434504 9876 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40003
I20240502 14:01:09.434759 9876 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:09.435252 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.72:0
--local_ip_for_outbound_sockets=127.24.66.72
--webserver_interface=127.24.66.72
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40003
--builtin_ntp_servers=127.24.66.84:34375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:01:09.435307 9876 heartbeater.cc:511] Master 127.24.66.126:40003 requested a full tablet report, sending...
I20240502 14:01:09.497539 8896 ts_manager.cc:194] Registered new tserver with Master: 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:09.498325 8896 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.71:35623
I20240502 14:01:09.502737 9216 heartbeater.cc:503] Master 127.24.66.126:40003 was elected leader, sending a full tablet report...
W20240502 14:01:09.557874 9879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:09.558095 9879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:09.558167 9879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:09.561321 9879 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:09.561409 9879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:09.561529 9879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.72
I20240502 14:01:09.565191 9879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.72:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.24.66.72
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40003
--never_fsync=true
--heap_profile_path=/tmp/kudu.9879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.72
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:09.566042 9879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:09.567116 9879 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:09.575850 9885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:09.576439 9886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:09.577054 9888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:09.577754 9879 server_base.cc:1034] running on GCE node
I20240502 14:01:09.578101 9879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:09.578619 9879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:09.579761 9879 hybrid_clock.cc:648] HybridClock initialized: now 1714658469579740 us; error 37 us; skew 500 ppm
I20240502 14:01:09.581746 9879 webserver.cc:460] Webserver started at http://127.24.66.72:36129/ using document root <none> and password file <none>
I20240502 14:01:09.582283 9879 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:09.582336 9879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:09.582491 9879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:01:09.584093 9879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "e01cf117ef5d446590c06dd1a6ab7ef7"
format_stamp: "Formatted at 2024-05-02 14:01:09 on dist-test-slave-5npg"
I20240502 14:01:09.584566 9879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "e01cf117ef5d446590c06dd1a6ab7ef7"
format_stamp: "Formatted at 2024-05-02 14:01:09 on dist-test-slave-5npg"
I20240502 14:01:09.588090 9879 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20240502 14:01:09.590265 9894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:09.591329 9879 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:09.591562 9879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "e01cf117ef5d446590c06dd1a6ab7ef7"
format_stamp: "Formatted at 2024-05-02 14:01:09 on dist-test-slave-5npg"
I20240502 14:01:09.591683 9879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:09.607685 9879 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:09.608270 9879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:09.608443 9879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:09.609016 9879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:09.609972 9879 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:09.610020 9879 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:09.610075 9879 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:09.610100 9879 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:09.619071 9879 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.72:35005
I20240502 14:01:09.619091 10007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.72:35005 every 8 connection(s)
I20240502 14:01:09.620041 9879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 14:01:09.621076 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 9879
I20240502 14:01:09.621186 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240502 14:01:09.623327 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.73:0
--local_ip_for_outbound_sockets=127.24.66.73
--webserver_interface=127.24.66.73
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40003
--builtin_ntp_servers=127.24.66.84:34375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:01:09.630710 10008 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40003
I20240502 14:01:09.631000 10008 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:09.631772 10008 heartbeater.cc:511] Master 127.24.66.126:40003 requested a full tablet report, sending...
I20240502 14:01:09.666612 8896 ts_manager.cc:194] Registered new tserver with Master: e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72:35005)
I20240502 14:01:09.667667 8896 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.72:44633
I20240502 14:01:09.688834 9348 heartbeater.cc:503] Master 127.24.66.126:40003 was elected leader, sending a full tablet report...
W20240502 14:01:09.739020 10011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:09.739239 10011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:09.739308 10011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:09.742431 10011 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:09.742501 10011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:09.742599 10011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.73
I20240502 14:01:09.746219 10011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.73:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--webserver_interface=127.24.66.73
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40003
--never_fsync=true
--heap_profile_path=/tmp/kudu.10011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.73
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:09.747066 10011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:09.748185 10011 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:09.756990 10017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:09.757589 10020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:09.758267 10018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:09.759038 10011 server_base.cc:1034] running on GCE node
I20240502 14:01:09.759421 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:09.760118 10011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:09.761276 10011 hybrid_clock.cc:648] HybridClock initialized: now 1714658469761256 us; error 37 us; skew 500 ppm
I20240502 14:01:09.763219 10011 webserver.cc:460] Webserver started at http://127.24.66.73:35739/ using document root <none> and password file <none>
I20240502 14:01:09.763784 10011 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:09.763844 10011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:09.764025 10011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:01:09.765806 10011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/data/instance:
uuid: "304274b88d234198a3892e38c05014a5"
format_stamp: "Formatted at 2024-05-02 14:01:09 on dist-test-slave-5npg"
I20240502 14:01:09.766290 10011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/wal/instance:
uuid: "304274b88d234198a3892e38c05014a5"
format_stamp: "Formatted at 2024-05-02 14:01:09 on dist-test-slave-5npg"
I20240502 14:01:09.769649 10011 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 14:01:09.771872 10026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:09.772989 10011 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:09.773082 10011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/wal
uuid: "304274b88d234198a3892e38c05014a5"
format_stamp: "Formatted at 2024-05-02 14:01:09 on dist-test-slave-5npg"
I20240502 14:01:09.773155 10011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:09.795053 10011 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:09.795662 10011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:09.795794 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:09.796388 10011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:09.797374 10011 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:09.797437 10011 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:09.797482 10011 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:09.797513 10011 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:09.805609 10011 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.73:42559
I20240502 14:01:09.805639 10139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.73:42559 every 8 connection(s)
I20240502 14:01:09.806514 10011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/data/info.pb
I20240502 14:01:09.809487 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 10011
I20240502 14:01:09.809618 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/wal/instance
I20240502 14:01:09.811898 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.74:0
--local_ip_for_outbound_sockets=127.24.66.74
--webserver_interface=127.24.66.74
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40003
--builtin_ntp_servers=127.24.66.84:34375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:01:09.816123 10140 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40003
I20240502 14:01:09.816385 10140 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:09.816987 10140 heartbeater.cc:511] Master 127.24.66.126:40003 requested a full tablet report, sending...
I20240502 14:01:09.857594 8896 ts_manager.cc:194] Registered new tserver with Master: 304274b88d234198a3892e38c05014a5 (127.24.66.73:42559)
I20240502 14:01:09.858361 8896 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.73:47335
I20240502 14:01:09.888489 9480 heartbeater.cc:503] Master 127.24.66.126:40003 was elected leader, sending a full tablet report...
W20240502 14:01:09.925555 10143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:09.925760 10143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:09.925833 10143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:09.930202 10143 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:09.930267 10143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:09.930354 10143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.74
I20240502 14:01:09.934134 10143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.74:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/data/info.pb
--webserver_interface=127.24.66.74
--webserver_port=0
--tserver_master_addrs=127.24.66.126:40003
--never_fsync=true
--heap_profile_path=/tmp/kudu.10143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.74
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:09.935007 10143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:09.936153 10143 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:09.944715 10152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:09.945353 10150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:09.945976 10149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:09.946655 10143 server_base.cc:1034] running on GCE node
I20240502 14:01:09.947002 10143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:09.947525 10143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:09.948671 10143 hybrid_clock.cc:648] HybridClock initialized: now 1714658469948660 us; error 25 us; skew 500 ppm
I20240502 14:01:09.950399 10143 webserver.cc:460] Webserver started at http://127.24.66.74:40941/ using document root <none> and password file <none>
I20240502 14:01:09.950932 10143 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:09.950990 10143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:09.951175 10143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 14:01:09.952821 10143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/data/instance:
uuid: "535b19a469c34efaaf49ca1cc2e1e5fc"
format_stamp: "Formatted at 2024-05-02 14:01:09 on dist-test-slave-5npg"
I20240502 14:01:09.953261 10143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/wal/instance:
uuid: "535b19a469c34efaaf49ca1cc2e1e5fc"
format_stamp: "Formatted at 2024-05-02 14:01:09 on dist-test-slave-5npg"
I20240502 14:01:09.956316 10143 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 14:01:09.958353 10158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:09.959378 10143 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20240502 14:01:09.959484 10143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/wal
uuid: "535b19a469c34efaaf49ca1cc2e1e5fc"
format_stamp: "Formatted at 2024-05-02 14:01:09 on dist-test-slave-5npg"
I20240502 14:01:09.959563 10143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:09.971740 10143 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:09.972266 10143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:09.972448 10143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:09.972944 10143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:09.973785 10143 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:09.973845 10143 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:09.973901 10143 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:09.973929 10143 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:09.982321 10143 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.74:34421
I20240502 14:01:09.982365 10271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.74:34421 every 8 connection(s)
I20240502 14:01:09.983160 10143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/data/info.pb
I20240502 14:01:09.988212 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 10143
I20240502 14:01:09.988368 24841 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/wal/instance
I20240502 14:01:09.991525 10272 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40003
I20240502 14:01:09.991696 10272 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:09.992122 10272 heartbeater.cc:511] Master 127.24.66.126:40003 requested a full tablet report, sending...
I20240502 14:01:10.027834 8896 ts_manager.cc:194] Registered new tserver with Master: 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74:34421)
I20240502 14:01:10.028556 8896 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.74:34261
I20240502 14:01:10.038396 24841 external_mini_cluster.cc:930] 10 TS(s) registered with all masters
I20240502 14:01:10.048229 24841 external_mini_cluster.cc:930] 10 TS(s) registered with all masters
I20240502 14:01:10.048306 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 10011
I20240502 14:01:10.054701 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 9879
I20240502 14:01:10.060863 24841 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu with pid 10143
I20240502 14:01:10.085568 9612 heartbeater.cc:503] Master 127.24.66.126:40003 was elected leader, sending a full tablet report...
I20240502 14:01:10.281046 9744 heartbeater.cc:503] Master 127.24.66.126:40003 was elected leader, sending a full tablet report...
I20240502 14:01:10.500641 9876 heartbeater.cc:503] Master 127.24.66.126:40003 was elected leader, sending a full tablet report...
I20240502 14:01:13.818131 8896 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46030:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240502 14:01:13.821430 8896 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_0 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240502 14:01:13.841356 9149 tablet_service.cc:1467] Processing CreateTablet for tablet 189b91e40d604414ab0497058929777c (DEFAULT_TABLE table=rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.842439 9149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 189b91e40d604414ab0497058929777c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.844561 9810 tablet_service.cc:1467] Processing CreateTablet for tablet bfa2e6af0fd544fa8bfdaefedabb5e68 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.845438 9810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfa2e6af0fd544fa8bfdaefedabb5e68. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.846501 9678 tablet_service.cc:1467] Processing CreateTablet for tablet bfa2e6af0fd544fa8bfdaefedabb5e68 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.847000 9676 tablet_service.cc:1467] Processing CreateTablet for tablet b1548ca1bcd24d288d30117baea53c12 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.847333 9678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfa2e6af0fd544fa8bfdaefedabb5e68. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.847489 9677 tablet_service.cc:1467] Processing CreateTablet for tablet 189b91e40d604414ab0497058929777c (DEFAULT_TABLE table=rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.847733 9677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 189b91e40d604414ab0497058929777c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.848444 9676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1548ca1bcd24d288d30117baea53c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.849079 9282 tablet_service.cc:1467] Processing CreateTablet for tablet bfa2e6af0fd544fa8bfdaefedabb5e68 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.849891 9282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfa2e6af0fd544fa8bfdaefedabb5e68. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.850072 9546 tablet_service.cc:1467] Processing CreateTablet for tablet bfa2e6af0fd544fa8bfdaefedabb5e68 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.850836 9546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfa2e6af0fd544fa8bfdaefedabb5e68. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.852600 9545 tablet_service.cc:1467] Processing CreateTablet for tablet 189b91e40d604414ab0497058929777c (DEFAULT_TABLE table=rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.852592 9281 tablet_service.cc:1467] Processing CreateTablet for tablet 189b91e40d604414ab0497058929777c (DEFAULT_TABLE table=rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.852818 9281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 189b91e40d604414ab0497058929777c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.852818 9545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 189b91e40d604414ab0497058929777c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.853338 9544 tablet_service.cc:1467] Processing CreateTablet for tablet b1548ca1bcd24d288d30117baea53c12 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.853569 9544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1548ca1bcd24d288d30117baea53c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.854646 9148 tablet_service.cc:1467] Processing CreateTablet for tablet b1548ca1bcd24d288d30117baea53c12 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.854892 9148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1548ca1bcd24d288d30117baea53c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.856513 9808 tablet_service.cc:1467] Processing CreateTablet for tablet b1548ca1bcd24d288d30117baea53c12 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.856819 9808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1548ca1bcd24d288d30117baea53c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.857964 10300 tablet_bootstrap.cc:492] T 189b91e40d604414ab0497058929777c P d83af1bc823a4f2ca531f0d6825f8e16: Bootstrap starting.
I20240502 14:01:13.860390 10300 tablet_bootstrap.cc:654] T 189b91e40d604414ab0497058929777c P d83af1bc823a4f2ca531f0d6825f8e16: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.860672 9280 tablet_service.cc:1467] Processing CreateTablet for tablet b1548ca1bcd24d288d30117baea53c12 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.860944 9280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1548ca1bcd24d288d30117baea53c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.861279 10300 log.cc:824] T 189b91e40d604414ab0497058929777c P d83af1bc823a4f2ca531f0d6825f8e16: Log is configured to *not* fsync() on all Append() calls
I20240502 14:01:13.861789 9017 tablet_service.cc:1467] Processing CreateTablet for tablet 189b91e40d604414ab0497058929777c (DEFAULT_TABLE table=rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.862287 9016 tablet_service.cc:1467] Processing CreateTablet for tablet b1548ca1bcd24d288d30117baea53c12 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.862470 9017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 189b91e40d604414ab0497058929777c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.862794 9016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1548ca1bcd24d288d30117baea53c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.861114 9018 tablet_service.cc:1467] Processing CreateTablet for tablet bfa2e6af0fd544fa8bfdaefedabb5e68 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.863612 9412 tablet_service.cc:1467] Processing CreateTablet for tablet b1548ca1bcd24d288d30117baea53c12 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.864157 9018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfa2e6af0fd544fa8bfdaefedabb5e68. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.864610 9150 tablet_service.cc:1467] Processing CreateTablet for tablet bfa2e6af0fd544fa8bfdaefedabb5e68 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.864919 9150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfa2e6af0fd544fa8bfdaefedabb5e68. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.865188 9414 tablet_service.cc:1467] Processing CreateTablet for tablet bfa2e6af0fd544fa8bfdaefedabb5e68 (DEFAULT_TABLE table=rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.865697 9413 tablet_service.cc:1467] Processing CreateTablet for tablet 189b91e40d604414ab0497058929777c (DEFAULT_TABLE table=rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.866066 9809 tablet_service.cc:1467] Processing CreateTablet for tablet 189b91e40d604414ab0497058929777c (DEFAULT_TABLE table=rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:13.866370 9809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 189b91e40d604414ab0497058929777c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.869892 10303 tablet_bootstrap.cc:492] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap starting.
I20240502 14:01:13.870416 9412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1548ca1bcd24d288d30117baea53c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.870889 9413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 189b91e40d604414ab0497058929777c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.871853 9414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfa2e6af0fd544fa8bfdaefedabb5e68. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:13.872969 10299 tablet_bootstrap.cc:492] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288: Bootstrap starting.
I20240502 14:01:13.873952 10301 tablet_bootstrap.cc:492] T 189b91e40d604414ab0497058929777c P ccfb5bbc1bb54793812d11170ac2d25f: Bootstrap starting.
I20240502 14:01:13.875046 10299 tablet_bootstrap.cc:654] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.875869 10299 log.cc:824] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288: Log is configured to *not* fsync() on all Append() calls
I20240502 14:01:13.876196 10301 tablet_bootstrap.cc:654] T 189b91e40d604414ab0497058929777c P ccfb5bbc1bb54793812d11170ac2d25f: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.877066 10301 log.cc:824] T 189b91e40d604414ab0497058929777c P ccfb5bbc1bb54793812d11170ac2d25f: Log is configured to *not* fsync() on all Append() calls
I20240502 14:01:13.877087 10303 tablet_bootstrap.cc:654] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.877262 10306 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:13.877851 10304 tablet_bootstrap.cc:492] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap starting.
I20240502 14:01:13.877851 10303 log.cc:824] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca: Log is configured to *not* fsync() on all Append() calls
I20240502 14:01:13.878562 10305 tablet_bootstrap.cc:492] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b: Bootstrap starting.
I20240502 14:01:13.879030 10306 tablet_bootstrap.cc:654] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.879731 10306 log.cc:824] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de: Log is configured to *not* fsync() on all Append() calls
I20240502 14:01:13.879868 10304 tablet_bootstrap.cc:654] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.880242 10299 tablet_bootstrap.cc:492] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288: No bootstrap required, opened a new log
I20240502 14:01:13.880314 10305 tablet_bootstrap.cc:654] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.880725 10304 log.cc:824] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d: Log is configured to *not* fsync() on all Append() calls
I20240502 14:01:13.881009 10305 log.cc:824] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b: Log is configured to *not* fsync() on all Append() calls
I20240502 14:01:13.881278 10306 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de: No bootstrap required, opened a new log
I20240502 14:01:13.881425 10299 ts_tablet_manager.cc:1367] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20240502 14:01:13.881443 10306 ts_tablet_manager.cc:1367] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240502 14:01:13.881500 10300 tablet_bootstrap.cc:492] T 189b91e40d604414ab0497058929777c P d83af1bc823a4f2ca531f0d6825f8e16: No bootstrap required, opened a new log
I20240502 14:01:13.881673 10300 ts_tablet_manager.cc:1367] T 189b91e40d604414ab0497058929777c P d83af1bc823a4f2ca531f0d6825f8e16: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.003s
I20240502 14:01:13.882802 10301 tablet_bootstrap.cc:492] T 189b91e40d604414ab0497058929777c P ccfb5bbc1bb54793812d11170ac2d25f: No bootstrap required, opened a new log
I20240502 14:01:13.882802 10303 tablet_bootstrap.cc:492] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca: No bootstrap required, opened a new log
I20240502 14:01:13.882952 10303 ts_tablet_manager.cc:1367] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20240502 14:01:13.883033 10301 ts_tablet_manager.cc:1367] T 189b91e40d604414ab0497058929777c P ccfb5bbc1bb54793812d11170ac2d25f: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20240502 14:01:13.885118 10300 raft_consensus.cc:348] T 189b91e40d604414ab0497058929777c P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.885161 10306 raft_consensus.cc:348] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } }
I20240502 14:01:13.885391 10300 raft_consensus.cc:374] T 189b91e40d604414ab0497058929777c P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.885391 10306 raft_consensus.cc:374] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.885447 10300 raft_consensus.cc:729] T 189b91e40d604414ab0497058929777c P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83af1bc823a4f2ca531f0d6825f8e16, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.885447 10306 raft_consensus.cc:729] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.885919 10306 consensus_queue.cc:260] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } }
I20240502 14:01:13.885919 10300 consensus_queue.cc:260] T 189b91e40d604414ab0497058929777c P d83af1bc823a4f2ca531f0d6825f8e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.886737 10306 ts_tablet_manager.cc:1398] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20240502 14:01:13.886737 10300 ts_tablet_manager.cc:1398] T 189b91e40d604414ab0497058929777c P d83af1bc823a4f2ca531f0d6825f8e16: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20240502 14:01:13.886638 10303 raft_consensus.cc:348] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.886842 10303 raft_consensus.cc:374] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.886899 10303 raft_consensus.cc:729] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.887063 10306 tablet_bootstrap.cc:492] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:13.887723 10301 raft_consensus.cc:348] T 189b91e40d604414ab0497058929777c P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.887918 10300 tablet_bootstrap.cc:492] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16: Bootstrap starting.
I20240502 14:01:13.888006 10306 tablet_bootstrap.cc:654] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.888950 10300 tablet_bootstrap.cc:654] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.889063 10301 raft_consensus.cc:374] T 189b91e40d604414ab0497058929777c P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.889169 10301 raft_consensus.cc:729] T 189b91e40d604414ab0497058929777c P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.889571 10301 consensus_queue.cc:260] T 189b91e40d604414ab0497058929777c P ccfb5bbc1bb54793812d11170ac2d25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.889571 10303 consensus_queue.cc:260] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.890355 10303 ts_tablet_manager.cc:1398] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20240502 14:01:13.890692 10303 tablet_bootstrap.cc:492] T 189b91e40d604414ab0497058929777c P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap starting.
I20240502 14:01:13.890326 10306 tablet_bootstrap.cc:492] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de: No bootstrap required, opened a new log
I20240502 14:01:13.891181 10306 ts_tablet_manager.cc:1367] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:01:13.891396 10301 ts_tablet_manager.cc:1398] T 189b91e40d604414ab0497058929777c P ccfb5bbc1bb54793812d11170ac2d25f: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20240502 14:01:13.891538 10303 tablet_bootstrap.cc:654] T 189b91e40d604414ab0497058929777c P d0f45862d7f14ce1976c1a37842f25ca: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.891898 10301 tablet_bootstrap.cc:492] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f: Bootstrap starting.
I20240502 14:01:13.892742 10303 tablet_bootstrap.cc:492] T 189b91e40d604414ab0497058929777c P d0f45862d7f14ce1976c1a37842f25ca: No bootstrap required, opened a new log
I20240502 14:01:13.892833 10303 ts_tablet_manager.cc:1367] T 189b91e40d604414ab0497058929777c P d0f45862d7f14ce1976c1a37842f25ca: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:13.894531 10304 tablet_bootstrap.cc:492] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d: No bootstrap required, opened a new log
I20240502 14:01:13.894838 10305 tablet_bootstrap.cc:492] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b: No bootstrap required, opened a new log
I20240502 14:01:13.894994 10305 ts_tablet_manager.cc:1367] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.003s
I20240502 14:01:13.895118 10303 raft_consensus.cc:348] T 189b91e40d604414ab0497058929777c P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.895251 10303 raft_consensus.cc:374] T 189b91e40d604414ab0497058929777c P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.895284 10303 raft_consensus.cc:729] T 189b91e40d604414ab0497058929777c P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.894757 10304 ts_tablet_manager.cc:1367] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.000s
I20240502 14:01:13.895447 10303 consensus_queue.cc:260] T 189b91e40d604414ab0497058929777c P d0f45862d7f14ce1976c1a37842f25ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.891719 10306 raft_consensus.cc:348] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.896136 10306 raft_consensus.cc:374] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.896184 10306 raft_consensus.cc:729] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.896364 10306 consensus_queue.cc:260] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.896728 10306 ts_tablet_manager.cc:1398] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
W20240502 14:01:13.897097 9349 tablet.cc:2369] T 189b91e40d604414ab0497058929777c P ccfb5bbc1bb54793812d11170ac2d25f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 14:01:13.898085 10306 tablet_bootstrap.cc:492] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:13.899050 10306 tablet_bootstrap.cc:654] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.898910 10305 raft_consensus.cc:348] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.899129 10305 raft_consensus.cc:374] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.899186 10305 raft_consensus.cc:729] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.899621 10305 consensus_queue.cc:260] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.899744 10303 ts_tablet_manager.cc:1398] T 189b91e40d604414ab0497058929777c P d0f45862d7f14ce1976c1a37842f25ca: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:01:13.899924 10303 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap starting.
I20240502 14:01:13.900514 10305 ts_tablet_manager.cc:1398] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b: Time spent starting tablet: real 0.005s user 0.005s sys 0.002s
I20240502 14:01:13.900840 10303 tablet_bootstrap.cc:654] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.900842 10305 tablet_bootstrap.cc:492] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b: Bootstrap starting.
I20240502 14:01:13.901723 10305 tablet_bootstrap.cc:654] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.901777 10306 tablet_bootstrap.cc:492] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de: No bootstrap required, opened a new log
I20240502 14:01:13.901849 10306 ts_tablet_manager.cc:1367] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:01:13.901883 10303 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca: No bootstrap required, opened a new log
I20240502 14:01:13.901948 10303 ts_tablet_manager.cc:1367] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:13.902276 10306 raft_consensus.cc:348] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.902395 10306 raft_consensus.cc:374] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.902318 10303 raft_consensus.cc:348] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } }
I20240502 14:01:13.902426 10306 raft_consensus.cc:729] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.902446 10303 raft_consensus.cc:374] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.902479 10303 raft_consensus.cc:729] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.902635 10303 consensus_queue.cc:260] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } }
I20240502 14:01:13.902938 10300 tablet_bootstrap.cc:492] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16: No bootstrap required, opened a new log
I20240502 14:01:13.903012 10300 ts_tablet_manager.cc:1367] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 14:01:13.902952 10299 raft_consensus.cc:348] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.903168 10299 raft_consensus.cc:374] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.903223 10299 raft_consensus.cc:729] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa836edab8d4853a7ca47a23117e288, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.903460 10303 ts_tablet_manager.cc:1398] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:13.903493 10306 consensus_queue.cc:260] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.903620 10299 consensus_queue.cc:260] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.903828 10300 raft_consensus.cc:348] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.903949 10300 raft_consensus.cc:374] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.903983 10300 raft_consensus.cc:729] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83af1bc823a4f2ca531f0d6825f8e16, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.904006 10321 raft_consensus.cc:482] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:13.904103 10300 consensus_queue.cc:260] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.904134 10321 raft_consensus.cc:504] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.904204 10305 tablet_bootstrap.cc:492] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b: No bootstrap required, opened a new log
I20240502 14:01:13.904312 10305 ts_tablet_manager.cc:1367] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:01:13.904393 10300 ts_tablet_manager.cc:1398] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 14:01:13.904554 10300 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16: Bootstrap starting.
I20240502 14:01:13.904707 10301 tablet_bootstrap.cc:654] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.905460 10300 tablet_bootstrap.cc:654] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.905664 10321 leader_election.cc:288] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749), 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397)
I20240502 14:01:13.906296 10306 ts_tablet_manager.cc:1398] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:01:13.906509 10304 raft_consensus.cc:348] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.906726 10304 raft_consensus.cc:374] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.906782 10304 raft_consensus.cc:729] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9e4f2f668d48ea87d5aad4a2fb569d, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.907125 10304 consensus_queue.cc:260] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.907475 10305 raft_consensus.cc:348] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.907593 10305 raft_consensus.cc:374] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.907629 10305 raft_consensus.cc:729] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.907730 10305 consensus_queue.cc:260] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.907948 10304 ts_tablet_manager.cc:1398] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20240502 14:01:13.907996 10305 ts_tablet_manager.cc:1398] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b: Time spent starting tablet: real 0.004s user 0.000s sys 0.000s
I20240502 14:01:13.908280 10304 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap starting.
I20240502 14:01:13.908733 10299 ts_tablet_manager.cc:1398] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288: Time spent starting tablet: real 0.027s user 0.005s sys 0.000s
I20240502 14:01:13.909040 10299 tablet_bootstrap.cc:492] T 189b91e40d604414ab0497058929777c P efa836edab8d4853a7ca47a23117e288: Bootstrap starting.
I20240502 14:01:13.910085 10299 tablet_bootstrap.cc:654] T 189b91e40d604414ab0497058929777c P efa836edab8d4853a7ca47a23117e288: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.910192 10305 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b: Bootstrap starting.
I20240502 14:01:13.911379 10304 tablet_bootstrap.cc:654] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.913013 10305 tablet_bootstrap.cc:654] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.913388 10300 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16: No bootstrap required, opened a new log
I20240502 14:01:13.913477 10300 ts_tablet_manager.cc:1367] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 14:01:13.913954 10300 raft_consensus.cc:348] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } }
I20240502 14:01:13.914067 10300 raft_consensus.cc:374] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.914099 10300 raft_consensus.cc:729] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83af1bc823a4f2ca531f0d6825f8e16, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.914237 10300 consensus_queue.cc:260] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } }
I20240502 14:01:13.914309 10301 tablet_bootstrap.cc:492] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f: No bootstrap required, opened a new log
I20240502 14:01:13.914397 10301 ts_tablet_manager.cc:1367] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20240502 14:01:13.914980 10301 raft_consensus.cc:348] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.915099 10301 raft_consensus.cc:374] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.915138 10301 raft_consensus.cc:729] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.915272 10301 consensus_queue.cc:260] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.916167 10301 ts_tablet_manager.cc:1398] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:13.916364 10301 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f: Bootstrap starting.
I20240502 14:01:13.917368 10301 tablet_bootstrap.cc:654] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.917866 10305 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b: No bootstrap required, opened a new log
I20240502 14:01:13.917938 10305 ts_tablet_manager.cc:1367] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 14:01:13.918347 10305 raft_consensus.cc:348] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } }
I20240502 14:01:13.918464 10305 raft_consensus.cc:374] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.918490 10305 raft_consensus.cc:729] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.918605 10305 consensus_queue.cc:260] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } }
I20240502 14:01:13.918849 10305 ts_tablet_manager.cc:1398] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:13.919127 10299 tablet_bootstrap.cc:492] T 189b91e40d604414ab0497058929777c P efa836edab8d4853a7ca47a23117e288: No bootstrap required, opened a new log
I20240502 14:01:13.919221 10299 ts_tablet_manager.cc:1367] T 189b91e40d604414ab0497058929777c P efa836edab8d4853a7ca47a23117e288: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 14:01:13.919557 9170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "189b91e40d604414ab0497058929777c" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" is_pre_election: true
I20240502 14:01:13.919767 10300 ts_tablet_manager.cc:1398] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240502 14:01:13.919777 10299 raft_consensus.cc:348] T 189b91e40d604414ab0497058929777c P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.919840 9170 raft_consensus.cc:2454] T 189b91e40d604414ab0497058929777c P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:13.919895 10299 raft_consensus.cc:374] T 189b91e40d604414ab0497058929777c P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.919927 10299 raft_consensus.cc:729] T 189b91e40d604414ab0497058929777c P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa836edab8d4853a7ca47a23117e288, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.920046 10299 consensus_queue.cc:260] T 189b91e40d604414ab0497058929777c P efa836edab8d4853a7ca47a23117e288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.920362 10299 ts_tablet_manager.cc:1398] T 189b91e40d604414ab0497058929777c P efa836edab8d4853a7ca47a23117e288: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:01:13.920536 10299 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288: Bootstrap starting.
I20240502 14:01:13.921485 10299 tablet_bootstrap.cc:654] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:13.922654 10299 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288: No bootstrap required, opened a new log
I20240502 14:01:13.922659 10301 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f: No bootstrap required, opened a new log
I20240502 14:01:13.922739 10299 ts_tablet_manager.cc:1367] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:13.922739 10301 ts_tablet_manager.cc:1367] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 14:01:13.923141 10299 raft_consensus.cc:348] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } }
I20240502 14:01:13.923259 10299 raft_consensus.cc:374] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.923302 10299 raft_consensus.cc:729] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa836edab8d4853a7ca47a23117e288, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.923441 10299 consensus_queue.cc:260] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } }
I20240502 14:01:13.923830 10299 ts_tablet_manager.cc:1398] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:13.923887 10301 raft_consensus.cc:348] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } }
I20240502 14:01:13.924014 10301 raft_consensus.cc:374] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.924062 10301 raft_consensus.cc:729] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.924229 10301 consensus_queue.cc:260] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } }
I20240502 14:01:13.924562 10301 ts_tablet_manager.cc:1398] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:13.924711 10304 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d: No bootstrap required, opened a new log
I20240502 14:01:13.924801 10304 ts_tablet_manager.cc:1367] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 14:01:13.925371 10304 raft_consensus.cc:348] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } }
I20240502 14:01:13.925510 10304 raft_consensus.cc:374] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.925544 10304 raft_consensus.cc:729] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9e4f2f668d48ea87d5aad4a2fb569d, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.925679 10304 consensus_queue.cc:260] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } }
I20240502 14:01:13.925977 10304 ts_tablet_manager.cc:1398] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:13.926154 10304 tablet_bootstrap.cc:492] T 189b91e40d604414ab0497058929777c P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap starting.
I20240502 14:01:13.926262 9698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "189b91e40d604414ab0497058929777c" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288" is_pre_election: true
I20240502 14:01:13.926522 9698 raft_consensus.cc:2454] T 189b91e40d604414ab0497058929777c P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:13.926599 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "189b91e40d604414ab0497058929777c" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" is_pre_election: true
I20240502 14:01:13.927099 10304 tablet_bootstrap.cc:654] T 189b91e40d604414ab0497058929777c P bc9e4f2f668d48ea87d5aad4a2fb569d: Neither blocks nor log segments found. Creating new log.
W20240502 14:01:13.927706 9765 leader_election.cc:341] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459): Illegal state: must be running to vote when last-logged opid is not known
I20240502 14:01:13.927965 10318 raft_consensus.cc:482] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:13.928121 10318 raft_consensus.cc:504] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } }
I20240502 14:01:13.929198 9434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "189b91e40d604414ab0497058929777c" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de" is_pre_election: true
I20240502 14:01:13.929299 9302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "189b91e40d604414ab0497058929777c" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" is_pre_election: true
I20240502 14:01:13.929488 9434 raft_consensus.cc:2454] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:13.929507 9302 raft_consensus.cc:2454] T 189b91e40d604414ab0497058929777c P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:13.929744 10318 leader_election.cc:288] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375)
I20240502 14:01:13.929826 10313 raft_consensus.cc:482] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:13.929911 10313 raft_consensus.cc:504] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
W20240502 14:01:13.930073 9877 tablet.cc:2369] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 14:01:13.930384 9763 leader_election.cc:302] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 0488b0e4ed994db19d40aed8c15f945b, ccfb5bbc1bb54793812d11170ac2d25f, d83af1bc823a4f2ca531f0d6825f8e16, efa836edab8d4853a7ca47a23117e288; no voters: bc9e4f2f668d48ea87d5aad4a2fb569d
I20240502 14:01:13.930622 10321 raft_consensus.cc:2781] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:13.930681 10321 raft_consensus.cc:482] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:13.930728 10321 raft_consensus.cc:3037] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.931411 10313 leader_election.cc:288] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:13.931916 10321 raft_consensus.cc:504] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.932545 10321 leader_election.cc:288] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 election: Requested vote from peers bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749), 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397)
I20240502 14:01:13.932577 9566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "189b91e40d604414ab0497058929777c" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca" is_pre_election: true
I20240502 14:01:13.932868 9566 raft_consensus.cc:2454] T 189b91e40d604414ab0497058929777c P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:13.933799 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "189b91e40d604414ab0497058929777c" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d"
I20240502 14:01:13.934036 9170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "189b91e40d604414ab0497058929777c" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16"
I20240502 14:01:13.934151 9170 raft_consensus.cc:3037] T 189b91e40d604414ab0497058929777c P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.934268 9302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "189b91e40d604414ab0497058929777c" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f"
I20240502 14:01:13.934391 9302 raft_consensus.cc:3037] T 189b91e40d604414ab0497058929777c P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.934688 9566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "189b91e40d604414ab0497058929777c" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca"
I20240502 14:01:13.934784 9566 raft_consensus.cc:3037] T 189b91e40d604414ab0497058929777c P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.935199 9170 raft_consensus.cc:2454] T 189b91e40d604414ab0497058929777c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:13.935468 9302 raft_consensus.cc:2454] T 189b91e40d604414ab0497058929777c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:13.935932 9698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "189b91e40d604414ab0497058929777c" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288"
I20240502 14:01:13.936050 9698 raft_consensus.cc:3037] T 189b91e40d604414ab0497058929777c P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.936267 9566 raft_consensus.cc:2454] T 189b91e40d604414ab0497058929777c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:13.936820 9765 leader_election.cc:302] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0488b0e4ed994db19d40aed8c15f945b, ccfb5bbc1bb54793812d11170ac2d25f, d0f45862d7f14ce1976c1a37842f25ca, d83af1bc823a4f2ca531f0d6825f8e16; no voters:
W20240502 14:01:13.936993 9765 leader_election.cc:341] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459): Illegal state: must be running to vote when last-logged opid is not known
I20240502 14:01:13.937212 9698 raft_consensus.cc:2454] T 189b91e40d604414ab0497058929777c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:13.937482 10321 raft_consensus.cc:2781] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:13.937621 9434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "189b91e40d604414ab0497058929777c" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de"
I20240502 14:01:13.937686 10321 raft_consensus.cc:686] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Becoming Leader. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Running, Role: LEADER
I20240502 14:01:13.937736 9434 raft_consensus.cc:3037] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.938019 10321 consensus_queue.cc:237] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.938997 9434 raft_consensus.cc:2454] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:13.940984 9698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfa2e6af0fd544fa8bfdaefedabb5e68" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288" is_pre_election: true
I20240502 14:01:13.941093 9698 raft_consensus.cc:2454] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 0.
I20240502 14:01:13.941704 9302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1548ca1bcd24d288d30117baea53c12" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" is_pre_election: true
I20240502 14:01:13.941831 9302 raft_consensus.cc:2454] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:13.942265 8896 catalog_manager.cc:5484] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71). New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:13.945453 9170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfa2e6af0fd544fa8bfdaefedabb5e68" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" is_pre_election: true
I20240502 14:01:13.945580 9170 raft_consensus.cc:2454] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 0.
I20240502 14:01:13.945955 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfa2e6af0fd544fa8bfdaefedabb5e68" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" is_pre_election: true
I20240502 14:01:13.946161 9038 raft_consensus.cc:2454] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 0.
I20240502 14:01:13.946578 9237 leader_election.cc:302] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: bc9e4f2f668d48ea87d5aad4a2fb569d, ccfb5bbc1bb54793812d11170ac2d25f, d83af1bc823a4f2ca531f0d6825f8e16, efa836edab8d4853a7ca47a23117e288; no voters:
I20240502 14:01:13.946820 10318 raft_consensus.cc:2781] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:13.946887 10318 raft_consensus.cc:482] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:13.946938 10318 raft_consensus.cc:3037] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.947885 10318 raft_consensus.cc:504] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } }
I20240502 14:01:13.948568 10318 leader_election.cc:288] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [CANDIDATE]: Term 1 election: Requested vote from peers d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375)
I20240502 14:01:13.949033 9698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfa2e6af0fd544fa8bfdaefedabb5e68" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288"
I20240502 14:01:13.949091 9170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1548ca1bcd24d288d30117baea53c12" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" is_pre_election: true
I20240502 14:01:13.949141 9698 raft_consensus.cc:3037] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.949195 9170 raft_consensus.cc:2454] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:13.949849 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfa2e6af0fd544fa8bfdaefedabb5e68" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d"
I20240502 14:01:13.949914 9434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfa2e6af0fd544fa8bfdaefedabb5e68" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de" is_pre_election: true
I20240502 14:01:13.949973 9038 raft_consensus.cc:3037] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.950037 9434 raft_consensus.cc:2454] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 0.
I20240502 14:01:13.950127 9698 raft_consensus.cc:2454] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 1.
I20240502 14:01:13.949923 9433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfa2e6af0fd544fa8bfdaefedabb5e68" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de"
I20240502 14:01:13.950259 9433 raft_consensus.cc:3037] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.951097 9038 raft_consensus.cc:2454] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 1.
I20240502 14:01:13.951270 9433 raft_consensus.cc:2454] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 1.
I20240502 14:01:13.951467 9170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfa2e6af0fd544fa8bfdaefedabb5e68" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16"
I20240502 14:01:13.951520 9566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfa2e6af0fd544fa8bfdaefedabb5e68" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca" is_pre_election: true
I20240502 14:01:13.951565 9170 raft_consensus.cc:3037] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.951647 9566 raft_consensus.cc:2454] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 0.
I20240502 14:01:13.951869 9565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfa2e6af0fd544fa8bfdaefedabb5e68" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca"
I20240502 14:01:13.951951 9565 raft_consensus.cc:3037] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.952169 9698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1548ca1bcd24d288d30117baea53c12" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288" is_pre_election: true
I20240502 14:01:13.952288 9698 raft_consensus.cc:2454] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:13.952569 9170 raft_consensus.cc:2454] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 1.
I20240502 14:01:13.952603 9370 leader_election.cc:302] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1a490f170c614ee4900ef16c3287c7de, ccfb5bbc1bb54793812d11170ac2d25f, d83af1bc823a4f2ca531f0d6825f8e16, efa836edab8d4853a7ca47a23117e288; no voters:
I20240502 14:01:13.952778 10304 tablet_bootstrap.cc:492] T 189b91e40d604414ab0497058929777c P bc9e4f2f668d48ea87d5aad4a2fb569d: No bootstrap required, opened a new log
I20240502 14:01:13.952859 10313 raft_consensus.cc:2781] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:13.952872 10304 ts_tablet_manager.cc:1367] T 189b91e40d604414ab0497058929777c P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.000s
I20240502 14:01:13.952922 10313 raft_consensus.cc:482] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:13.952957 10313 raft_consensus.cc:3037] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.953408 9237 leader_election.cc:302] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1a490f170c614ee4900ef16c3287c7de, bc9e4f2f668d48ea87d5aad4a2fb569d, ccfb5bbc1bb54793812d11170ac2d25f, efa836edab8d4853a7ca47a23117e288; no voters:
I20240502 14:01:13.953397 10304 raft_consensus.cc:348] T 189b91e40d604414ab0497058929777c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.953521 10304 raft_consensus.cc:374] T 189b91e40d604414ab0497058929777c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:13.953554 10304 raft_consensus.cc:729] T 189b91e40d604414ab0497058929777c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9e4f2f668d48ea87d5aad4a2fb569d, State: Initialized, Role: FOLLOWER
I20240502 14:01:13.953694 10304 consensus_queue.cc:260] T 189b91e40d604414ab0497058929777c P bc9e4f2f668d48ea87d5aad4a2fb569d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.953819 10313 raft_consensus.cc:504] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.954025 10304 ts_tablet_manager.cc:1398] T 189b91e40d604414ab0497058929777c P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:13.954242 10318 raft_consensus.cc:2781] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:13.954356 10318 raft_consensus.cc:686] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Becoming Leader. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Running, Role: LEADER
I20240502 14:01:13.954381 10313 leader_election.cc:288] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [CANDIDATE]: Term 1 election: Requested vote from peers bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:13.954494 9565 raft_consensus.cc:2454] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 1.
I20240502 14:01:13.954689 10318 consensus_queue.cc:237] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } }
I20240502 14:01:13.955870 9302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1548ca1bcd24d288d30117baea53c12" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f"
I20240502 14:01:13.955972 9302 raft_consensus.cc:3037] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.956871 9302 raft_consensus.cc:2454] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:13.956988 9698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1548ca1bcd24d288d30117baea53c12" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288"
I20240502 14:01:13.957048 9170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1548ca1bcd24d288d30117baea53c12" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16"
I20240502 14:01:13.957103 9698 raft_consensus.cc:3037] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.957171 9170 raft_consensus.cc:3037] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.957326 9566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1548ca1bcd24d288d30117baea53c12" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca"
I20240502 14:01:13.957438 9566 raft_consensus.cc:3037] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.958014 9698 raft_consensus.cc:2454] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:13.958060 9170 raft_consensus.cc:2454] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:13.958308 9566 raft_consensus.cc:2454] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
W20240502 14:01:13.958580 9217 tablet.cc:2369] T 189b91e40d604414ab0497058929777c P d83af1bc823a4f2ca531f0d6825f8e16: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 14:01:13.958926 9368 leader_election.cc:302] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1a490f170c614ee4900ef16c3287c7de, d0f45862d7f14ce1976c1a37842f25ca, d83af1bc823a4f2ca531f0d6825f8e16, efa836edab8d4853a7ca47a23117e288; no voters:
I20240502 14:01:13.959122 10313 raft_consensus.cc:2781] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:13.959215 9565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1548ca1bcd24d288d30117baea53c12" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca" is_pre_election: true
I20240502 14:01:13.959228 9037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1548ca1bcd24d288d30117baea53c12" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d"
I20240502 14:01:13.959319 9037 raft_consensus.cc:3037] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.959329 9565 raft_consensus.cc:2362] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1. Re-sending same reply.
I20240502 14:01:13.959329 10313 raft_consensus.cc:686] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Becoming Leader. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Running, Role: LEADER
I20240502 14:01:13.959664 10313 consensus_queue.cc:237] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:13.960261 9037 raft_consensus.cc:2454] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:13.960248 9829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1548ca1bcd24d288d30117baea53c12" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b"
I20240502 14:01:13.960433 9829 raft_consensus.cc:3037] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:13.960623 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1548ca1bcd24d288d30117baea53c12" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" is_pre_election: true
I20240502 14:01:13.960742 9038 raft_consensus.cc:2362] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1. Re-sending same reply.
I20240502 14:01:13.961108 9830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1548ca1bcd24d288d30117baea53c12" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b" is_pre_election: true
I20240502 14:01:13.961933 9829 raft_consensus.cc:2454] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:13.962648 8894 catalog_manager.cc:5484] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: term changed from 0 to 1, leader changed from <none> to ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67). New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:13.963773 8896 catalog_manager.cc:5484] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a490f170c614ee4900ef16c3287c7de (127.24.66.68). New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:13.964978 9829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfa2e6af0fd544fa8bfdaefedabb5e68" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b" is_pre_election: true
I20240502 14:01:13.965016 9830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfa2e6af0fd544fa8bfdaefedabb5e68" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b"
I20240502 14:01:13.965097 9829 raft_consensus.cc:2454] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 0.
W20240502 14:01:13.988524 9745 tablet.cc:2369] T 189b91e40d604414ab0497058929777c P efa836edab8d4853a7ca47a23117e288: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 14:01:14.018047 9085 tablet.cc:2369] T 189b91e40d604414ab0497058929777c P bc9e4f2f668d48ea87d5aad4a2fb569d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 14:01:14.029623 9613 tablet.cc:2369] T 189b91e40d604414ab0497058929777c P d0f45862d7f14ce1976c1a37842f25ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 14:01:14.083340 9481 tablet.cc:2369] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 14:01:14.088002 10364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:14.088234 10364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:14.091261 10364 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240502 14:01:14.185448 9829 raft_consensus.cc:3037] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.186681 9829 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.186861 9433 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.187135 9698 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.187661 10321 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.187664 10318 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.188457 10321 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.188711 9302 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.189203 10396 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.189333 9038 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.190085 9565 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.190819 9170 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.191922 10396 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.192011 9698 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.192222 10361 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.192773 9829 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.193409 9301 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.193883 10321 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.194087 9565 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.194249 9434 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.194957 9698 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.195335 10321 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.195287 10340 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.197840 9170 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.198235 9038 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.198248 10318 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.198704 9565 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.198951 10340 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.199229 10340 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.200003 10361 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.200312 10361 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.203465 10313 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.203949 10318 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.209280 9036 raft_consensus.cc:3037] T 189b91e40d604414ab0497058929777c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.210481 9036 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.210837 9168 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.212884 10321 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.213207 10321 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.232127 10414 mvcc.cc:205] Tried to move back new op lower bound from 7023241110263132160 to 7023241109254950912. Current Snapshot: MvccSnapshot[applied={T|T < 7023241110263132160}]
I20240502 14:01:14.232465 10406 mvcc.cc:205] Tried to move back new op lower bound from 7023241110263132160 to 7023241109254950912. Current Snapshot: MvccSnapshot[applied={T|T < 7023241110263132160}]
I20240502 14:01:14.248958 10440 mvcc.cc:205] Tried to move back new op lower bound from 7023241110263132160 to 7023241109254950912. Current Snapshot: MvccSnapshot[applied={T|T < 7023241110263132160 or (T in {7023241110263132160})}]
I20240502 14:01:14.255955 10428 mvcc.cc:205] Tried to move back new op lower bound from 7023241110255689728 to 7023241109339688960. Current Snapshot: MvccSnapshot[applied={T|T < 7023241110255689728}]
I20240502 14:01:14.265456 10417 mvcc.cc:205] Tried to move back new op lower bound from 7023241110266171392 to 7023241109343518720. Current Snapshot: MvccSnapshot[applied={T|T < 7023241110266171392}]
I20240502 14:01:14.271726 10400 mvcc.cc:205] Tried to move back new op lower bound from 7023241110255689728 to 7023241109339688960. Current Snapshot: MvccSnapshot[applied={T|T < 7023241110255689728}]
I20240502 14:01:14.297981 10430 mvcc.cc:205] Tried to move back new op lower bound from 7023241110255689728 to 7023241109339688960. Current Snapshot: MvccSnapshot[applied={T|T < 7023241110255689728}]
INSERT report
rows total: 2000
time total: 213.79 ms
time per row: 0.106895 ms
I20240502 14:01:14.356685 8892 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46030:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240502 14:01:14.357321 8892 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_1 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240502 14:01:14.362957 9412 tablet_service.cc:1467] Processing CreateTablet for tablet 39bef244549d4bb7a980bf7bc52a01b4 (DEFAULT_TABLE table=rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.363318 9412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39bef244549d4bb7a980bf7bc52a01b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.363463 9150 tablet_service.cc:1467] Processing CreateTablet for tablet 39bef244549d4bb7a980bf7bc52a01b4 (DEFAULT_TABLE table=rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.363719 9150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39bef244549d4bb7a980bf7bc52a01b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.363749 9282 tablet_service.cc:1467] Processing CreateTablet for tablet 39bef244549d4bb7a980bf7bc52a01b4 (DEFAULT_TABLE table=rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.364040 9282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39bef244549d4bb7a980bf7bc52a01b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.365489 9546 tablet_service.cc:1467] Processing CreateTablet for tablet 39bef244549d4bb7a980bf7bc52a01b4 (DEFAULT_TABLE table=rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.365758 9546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39bef244549d4bb7a980bf7bc52a01b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.365854 10306 tablet_bootstrap.cc:492] T 39bef244549d4bb7a980bf7bc52a01b4 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:14.365921 10300 tablet_bootstrap.cc:492] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16: Bootstrap starting.
I20240502 14:01:14.366539 10301 tablet_bootstrap.cc:492] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f: Bootstrap starting.
I20240502 14:01:14.366691 10300 tablet_bootstrap.cc:654] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.366691 10306 tablet_bootstrap.cc:654] T 39bef244549d4bb7a980bf7bc52a01b4 P 1a490f170c614ee4900ef16c3287c7de: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.367398 10301 tablet_bootstrap.cc:654] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.367455 9018 tablet_service.cc:1467] Processing CreateTablet for tablet 39bef244549d4bb7a980bf7bc52a01b4 (DEFAULT_TABLE table=rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.367754 9018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39bef244549d4bb7a980bf7bc52a01b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.367971 9544 tablet_service.cc:1467] Processing CreateTablet for tablet 50451fdbd43b491fa701daac5de5e1c8 (DEFAULT_TABLE table=rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.368196 9544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50451fdbd43b491fa701daac5de5e1c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.368134 9678 tablet_service.cc:1467] Processing CreateTablet for tablet 39bef244549d4bb7a980bf7bc52a01b4 (DEFAULT_TABLE table=rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.368279 9677 tablet_service.cc:1467] Processing CreateTablet for tablet 50451fdbd43b491fa701daac5de5e1c8 (DEFAULT_TABLE table=rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.368435 9678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39bef244549d4bb7a980bf7bc52a01b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.368549 9545 tablet_service.cc:1467] Processing CreateTablet for tablet 9eef89d7e129492bab305a1b5009aace (DEFAULT_TABLE table=rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.368605 9676 tablet_service.cc:1467] Processing CreateTablet for tablet 9eef89d7e129492bab305a1b5009aace (DEFAULT_TABLE table=rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.368748 9545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eef89d7e129492bab305a1b5009aace. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.368815 9676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eef89d7e129492bab305a1b5009aace. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.369926 10303 tablet_bootstrap.cc:492] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap starting.
I20240502 14:01:14.370525 10299 tablet_bootstrap.cc:492] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288: Bootstrap starting.
I20240502 14:01:14.370810 10303 tablet_bootstrap.cc:654] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.371346 10299 tablet_bootstrap.cc:654] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.371307 9280 tablet_service.cc:1467] Processing CreateTablet for tablet 9eef89d7e129492bab305a1b5009aace (DEFAULT_TABLE table=rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.371425 10304 tablet_bootstrap.cc:492] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap starting.
I20240502 14:01:14.371541 9280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eef89d7e129492bab305a1b5009aace. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.372220 10304 tablet_bootstrap.cc:654] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.372509 9282 tablet_service.cc:1467] Processing CreateTablet for tablet 50451fdbd43b491fa701daac5de5e1c8 (DEFAULT_TABLE table=rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.372725 9282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50451fdbd43b491fa701daac5de5e1c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.373418 9810 tablet_service.cc:1467] Processing CreateTablet for tablet 39bef244549d4bb7a980bf7bc52a01b4 (DEFAULT_TABLE table=rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.373551 9677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50451fdbd43b491fa701daac5de5e1c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.373701 9810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39bef244549d4bb7a980bf7bc52a01b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.373843 10301 tablet_bootstrap.cc:492] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f: No bootstrap required, opened a new log
I20240502 14:01:14.373919 10301 ts_tablet_manager.cc:1367] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 14:01:14.374358 10301 raft_consensus.cc:348] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.374505 10301 raft_consensus.cc:374] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.374547 10301 raft_consensus.cc:729] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.374661 10301 consensus_queue.cc:260] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.374935 10301 ts_tablet_manager.cc:1398] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:14.374980 10303 tablet_bootstrap.cc:492] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca: No bootstrap required, opened a new log
I20240502 14:01:14.375056 10303 ts_tablet_manager.cc:1367] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 14:01:14.375103 10301 tablet_bootstrap.cc:492] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f: Bootstrap starting.
I20240502 14:01:14.375509 10303 raft_consensus.cc:348] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.375630 10303 raft_consensus.cc:374] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.375669 10303 raft_consensus.cc:729] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.375792 10303 consensus_queue.cc:260] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.375946 10301 tablet_bootstrap.cc:654] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.376070 10303 ts_tablet_manager.cc:1398] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:14.376067 9808 tablet_service.cc:1467] Processing CreateTablet for tablet 50451fdbd43b491fa701daac5de5e1c8 (DEFAULT_TABLE table=rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.376255 10303 tablet_bootstrap.cc:492] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap starting.
I20240502 14:01:14.376308 9808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50451fdbd43b491fa701daac5de5e1c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.377158 10303 tablet_bootstrap.cc:654] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.377420 9148 tablet_service.cc:1467] Processing CreateTablet for tablet 9eef89d7e129492bab305a1b5009aace (DEFAULT_TABLE table=rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.377465 9150 tablet_service.cc:1467] Processing CreateTablet for tablet 50451fdbd43b491fa701daac5de5e1c8 (DEFAULT_TABLE table=rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.377677 9148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eef89d7e129492bab305a1b5009aace. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.377871 9150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50451fdbd43b491fa701daac5de5e1c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.378420 9809 tablet_service.cc:1467] Processing CreateTablet for tablet 9eef89d7e129492bab305a1b5009aace (DEFAULT_TABLE table=rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.378665 10299 tablet_bootstrap.cc:492] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288: No bootstrap required, opened a new log
I20240502 14:01:14.378675 9809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eef89d7e129492bab305a1b5009aace. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.378749 10299 ts_tablet_manager.cc:1367] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240502 14:01:14.379212 10299 raft_consensus.cc:348] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.379338 10299 raft_consensus.cc:374] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.379366 10299 raft_consensus.cc:729] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa836edab8d4853a7ca47a23117e288, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.379479 10299 consensus_queue.cc:260] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.379776 10299 ts_tablet_manager.cc:1398] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:14.379945 10299 tablet_bootstrap.cc:492] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288: Bootstrap starting.
I20240502 14:01:14.380496 10305 tablet_bootstrap.cc:492] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b: Bootstrap starting.
I20240502 14:01:14.380892 10299 tablet_bootstrap.cc:654] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.381299 10305 tablet_bootstrap.cc:654] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.382786 10301 tablet_bootstrap.cc:492] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f: No bootstrap required, opened a new log
I20240502 14:01:14.382869 10301 ts_tablet_manager.cc:1367] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:01:14.382872 10299 tablet_bootstrap.cc:492] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288: No bootstrap required, opened a new log
I20240502 14:01:14.382930 10305 tablet_bootstrap.cc:492] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b: No bootstrap required, opened a new log
I20240502 14:01:14.382951 10299 ts_tablet_manager.cc:1367] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 14:01:14.383014 10305 ts_tablet_manager.cc:1367] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 14:01:14.383266 10301 raft_consensus.cc:348] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.383366 10301 raft_consensus.cc:374] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.383400 10301 raft_consensus.cc:729] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.383343 10299 raft_consensus.cc:348] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.383406 10305 raft_consensus.cc:348] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.383513 10299 raft_consensus.cc:374] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.383518 10305 raft_consensus.cc:374] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.383558 10305 raft_consensus.cc:729] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.383558 10299 raft_consensus.cc:729] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa836edab8d4853a7ca47a23117e288, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.383544 10301 consensus_queue.cc:260] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.383673 10305 consensus_queue.cc:260] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.383673 10299 consensus_queue.cc:260] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.383859 10301 ts_tablet_manager.cc:1398] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:14.383929 10305 ts_tablet_manager.cc:1398] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:14.383939 10299 ts_tablet_manager.cc:1398] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:14.384006 10301 tablet_bootstrap.cc:492] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f: Bootstrap starting.
I20240502 14:01:14.384068 10305 tablet_bootstrap.cc:492] T 50451fdbd43b491fa701daac5de5e1c8 P 0488b0e4ed994db19d40aed8c15f945b: Bootstrap starting.
I20240502 14:01:14.384068 10299 tablet_bootstrap.cc:492] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288: Bootstrap starting.
I20240502 14:01:14.385236 10301 tablet_bootstrap.cc:654] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.386037 10299 tablet_bootstrap.cc:654] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.386725 9017 tablet_service.cc:1467] Processing CreateTablet for tablet 9eef89d7e129492bab305a1b5009aace (DEFAULT_TABLE table=rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.386919 10301 tablet_bootstrap.cc:492] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f: No bootstrap required, opened a new log
I20240502 14:01:14.386994 10301 ts_tablet_manager.cc:1367] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 14:01:14.387003 9017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eef89d7e129492bab305a1b5009aace. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.387140 9413 tablet_service.cc:1467] Processing CreateTablet for tablet 9eef89d7e129492bab305a1b5009aace (DEFAULT_TABLE table=rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.387436 9413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eef89d7e129492bab305a1b5009aace. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.387435 10301 raft_consensus.cc:348] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.387562 10301 raft_consensus.cc:374] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.387607 10301 raft_consensus.cc:729] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.387728 10301 consensus_queue.cc:260] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.388049 10301 ts_tablet_manager.cc:1398] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:14.388494 9018 tablet_service.cc:1467] Processing CreateTablet for tablet 50451fdbd43b491fa701daac5de5e1c8 (DEFAULT_TABLE table=rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.389766 10305 tablet_bootstrap.cc:654] T 50451fdbd43b491fa701daac5de5e1c8 P 0488b0e4ed994db19d40aed8c15f945b: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.390209 10303 tablet_bootstrap.cc:492] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca: No bootstrap required, opened a new log
I20240502 14:01:14.390307 10303 ts_tablet_manager.cc:1367] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 14:01:14.390852 10304 tablet_bootstrap.cc:492] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d: No bootstrap required, opened a new log
I20240502 14:01:14.390830 10303 raft_consensus.cc:348] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.390938 10304 ts_tablet_manager.cc:1367] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.002s
I20240502 14:01:14.390944 10303 raft_consensus.cc:374] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.390976 10303 raft_consensus.cc:729] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.391100 10303 consensus_queue.cc:260] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.391367 10303 ts_tablet_manager.cc:1398] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:14.391412 10304 raft_consensus.cc:348] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.391530 10304 raft_consensus.cc:374] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.391541 10299 tablet_bootstrap.cc:492] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288: No bootstrap required, opened a new log
I20240502 14:01:14.391552 10303 tablet_bootstrap.cc:492] T 50451fdbd43b491fa701daac5de5e1c8 P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap starting.
I20240502 14:01:14.391566 10304 raft_consensus.cc:729] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9e4f2f668d48ea87d5aad4a2fb569d, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.391620 10299 ts_tablet_manager.cc:1367] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:01:14.391692 10304 consensus_queue.cc:260] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.391975 10304 ts_tablet_manager.cc:1398] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:14.392048 10299 raft_consensus.cc:348] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.392174 10304 tablet_bootstrap.cc:492] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap starting.
I20240502 14:01:14.392169 10299 raft_consensus.cc:374] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.392212 10299 raft_consensus.cc:729] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa836edab8d4853a7ca47a23117e288, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.392365 10299 consensus_queue.cc:260] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.392702 10299 ts_tablet_manager.cc:1398] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:14.392717 10303 tablet_bootstrap.cc:654] T 50451fdbd43b491fa701daac5de5e1c8 P d0f45862d7f14ce1976c1a37842f25ca: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.394732 10300 tablet_bootstrap.cc:492] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16: No bootstrap required, opened a new log
I20240502 14:01:14.394858 10300 ts_tablet_manager.cc:1367] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.000s
I20240502 14:01:14.395275 10300 raft_consensus.cc:348] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.395366 10306 tablet_bootstrap.cc:492] T 39bef244549d4bb7a980bf7bc52a01b4 P 1a490f170c614ee4900ef16c3287c7de: No bootstrap required, opened a new log
I20240502 14:01:14.395382 10300 raft_consensus.cc:374] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.395418 10300 raft_consensus.cc:729] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83af1bc823a4f2ca531f0d6825f8e16, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.395447 10306 ts_tablet_manager.cc:1367] T 39bef244549d4bb7a980bf7bc52a01b4 P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.030s user 0.001s sys 0.000s
I20240502 14:01:14.395568 10300 consensus_queue.cc:260] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.395860 10300 ts_tablet_manager.cc:1398] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:14.395992 10300 tablet_bootstrap.cc:492] T 50451fdbd43b491fa701daac5de5e1c8 P d83af1bc823a4f2ca531f0d6825f8e16: Bootstrap starting.
I20240502 14:01:14.395907 10306 raft_consensus.cc:348] T 39bef244549d4bb7a980bf7bc52a01b4 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.396024 10306 raft_consensus.cc:374] T 39bef244549d4bb7a980bf7bc52a01b4 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.396057 10306 raft_consensus.cc:729] T 39bef244549d4bb7a980bf7bc52a01b4 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.396183 10306 consensus_queue.cc:260] T 39bef244549d4bb7a980bf7bc52a01b4 P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.396525 10306 ts_tablet_manager.cc:1398] T 39bef244549d4bb7a980bf7bc52a01b4 P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:14.396704 10306 tablet_bootstrap.cc:492] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:14.396842 10300 tablet_bootstrap.cc:654] T 50451fdbd43b491fa701daac5de5e1c8 P d83af1bc823a4f2ca531f0d6825f8e16: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.397809 10306 tablet_bootstrap.cc:654] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.397892 10300 tablet_bootstrap.cc:492] T 50451fdbd43b491fa701daac5de5e1c8 P d83af1bc823a4f2ca531f0d6825f8e16: No bootstrap required, opened a new log
I20240502 14:01:14.397980 10300 ts_tablet_manager.cc:1367] T 50451fdbd43b491fa701daac5de5e1c8 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:01:14.398705 10300 raft_consensus.cc:348] T 50451fdbd43b491fa701daac5de5e1c8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.398837 10300 raft_consensus.cc:374] T 50451fdbd43b491fa701daac5de5e1c8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.398878 10300 raft_consensus.cc:729] T 50451fdbd43b491fa701daac5de5e1c8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83af1bc823a4f2ca531f0d6825f8e16, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.399021 10300 consensus_queue.cc:260] T 50451fdbd43b491fa701daac5de5e1c8 P d83af1bc823a4f2ca531f0d6825f8e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.399066 9412 tablet_service.cc:1467] Processing CreateTablet for tablet 50451fdbd43b491fa701daac5de5e1c8 (DEFAULT_TABLE table=rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.399314 10300 ts_tablet_manager.cc:1398] T 50451fdbd43b491fa701daac5de5e1c8 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:14.399333 9412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50451fdbd43b491fa701daac5de5e1c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.399498 10300 tablet_bootstrap.cc:492] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16: Bootstrap starting.
I20240502 14:01:14.399765 9018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50451fdbd43b491fa701daac5de5e1c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.399962 10306 tablet_bootstrap.cc:492] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de: No bootstrap required, opened a new log
I20240502 14:01:14.400146 10306 ts_tablet_manager.cc:1367] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:01:14.400403 10339 raft_consensus.cc:482] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:14.400530 10339 raft_consensus.cc:504] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.400820 10300 tablet_bootstrap.cc:654] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.400931 10306 raft_consensus.cc:348] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.401124 10306 raft_consensus.cc:374] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.401222 10304 tablet_bootstrap.cc:654] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.401271 10306 raft_consensus.cc:729] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.401597 10306 consensus_queue.cc:260] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.401948 10306 ts_tablet_manager.cc:1398] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:14.402227 10339 leader_election.cc:288] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679), bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:14.402642 10304 tablet_bootstrap.cc:492] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d: No bootstrap required, opened a new log
I20240502 14:01:14.402716 10304 ts_tablet_manager.cc:1367] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 14:01:14.403311 10304 raft_consensus.cc:348] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.403450 10304 raft_consensus.cc:374] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.403491 10304 raft_consensus.cc:729] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9e4f2f668d48ea87d5aad4a2fb569d, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.403626 10304 consensus_queue.cc:260] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.404006 10304 ts_tablet_manager.cc:1398] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:14.406314 10306 tablet_bootstrap.cc:492] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:14.407014 10303 tablet_bootstrap.cc:492] T 50451fdbd43b491fa701daac5de5e1c8 P d0f45862d7f14ce1976c1a37842f25ca: No bootstrap required, opened a new log
I20240502 14:01:14.407105 10303 ts_tablet_manager.cc:1367] T 50451fdbd43b491fa701daac5de5e1c8 P d0f45862d7f14ce1976c1a37842f25ca: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 14:01:14.407135 10305 tablet_bootstrap.cc:492] T 50451fdbd43b491fa701daac5de5e1c8 P 0488b0e4ed994db19d40aed8c15f945b: No bootstrap required, opened a new log
I20240502 14:01:14.407267 10305 ts_tablet_manager.cc:1367] T 50451fdbd43b491fa701daac5de5e1c8 P 0488b0e4ed994db19d40aed8c15f945b: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20240502 14:01:14.407315 10306 tablet_bootstrap.cc:654] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.407624 10303 raft_consensus.cc:348] T 50451fdbd43b491fa701daac5de5e1c8 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.407744 10303 raft_consensus.cc:374] T 50451fdbd43b491fa701daac5de5e1c8 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.407783 10303 raft_consensus.cc:729] T 50451fdbd43b491fa701daac5de5e1c8 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.407753 10305 raft_consensus.cc:348] T 50451fdbd43b491fa701daac5de5e1c8 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.407879 10305 raft_consensus.cc:374] T 50451fdbd43b491fa701daac5de5e1c8 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.407899 10304 tablet_bootstrap.cc:492] T 50451fdbd43b491fa701daac5de5e1c8 P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap starting.
I20240502 14:01:14.407913 10305 raft_consensus.cc:729] T 50451fdbd43b491fa701daac5de5e1c8 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.407908 10303 consensus_queue.cc:260] T 50451fdbd43b491fa701daac5de5e1c8 P d0f45862d7f14ce1976c1a37842f25ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.408043 10305 consensus_queue.cc:260] T 50451fdbd43b491fa701daac5de5e1c8 P 0488b0e4ed994db19d40aed8c15f945b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.408193 10303 ts_tablet_manager.cc:1398] T 50451fdbd43b491fa701daac5de5e1c8 P d0f45862d7f14ce1976c1a37842f25ca: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:14.408429 10305 ts_tablet_manager.cc:1398] T 50451fdbd43b491fa701daac5de5e1c8 P 0488b0e4ed994db19d40aed8c15f945b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:01:14.408627 10305 tablet_bootstrap.cc:492] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b: Bootstrap starting.
I20240502 14:01:14.409595 10304 tablet_bootstrap.cc:654] T 50451fdbd43b491fa701daac5de5e1c8 P bc9e4f2f668d48ea87d5aad4a2fb569d: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.409658 10305 tablet_bootstrap.cc:654] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.410454 10437 raft_consensus.cc:482] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:14.410573 10437 raft_consensus.cc:504] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.411275 10437 leader_election.cc:288] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679), bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837)
I20240502 14:01:14.411437 9433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50451fdbd43b491fa701daac5de5e1c8" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de" is_pre_election: true
I20240502 14:01:14.411748 9565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50451fdbd43b491fa701daac5de5e1c8" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca" is_pre_election: true
I20240502 14:01:14.411863 9565 raft_consensus.cc:2454] T 50451fdbd43b491fa701daac5de5e1c8 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 0.
I20240502 14:01:14.412591 9698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50451fdbd43b491fa701daac5de5e1c8" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288" is_pre_election: true
I20240502 14:01:14.412657 9829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50451fdbd43b491fa701daac5de5e1c8" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b" is_pre_election: true
I20240502 14:01:14.412756 9698 raft_consensus.cc:2454] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 0.
I20240502 14:01:14.412777 9829 raft_consensus.cc:2454] T 50451fdbd43b491fa701daac5de5e1c8 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 0.
I20240502 14:01:14.413054 9235 leader_election.cc:302] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0488b0e4ed994db19d40aed8c15f945b, ccfb5bbc1bb54793812d11170ac2d25f, d0f45862d7f14ce1976c1a37842f25ca, efa836edab8d4853a7ca47a23117e288; no voters:
W20240502 14:01:14.413478 9237 leader_election.cc:341] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907): Illegal state: must be running to vote when last-logged opid is not known
I20240502 14:01:14.414011 9302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50451fdbd43b491fa701daac5de5e1c8" candidate_uuid: "efa836edab8d4853a7ca47a23117e288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" is_pre_election: true
I20240502 14:01:14.414000 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50451fdbd43b491fa701daac5de5e1c8" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" is_pre_election: true
I20240502 14:01:14.414132 9302 raft_consensus.cc:2454] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa836edab8d4853a7ca47a23117e288 in term 0.
I20240502 14:01:14.414304 9170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50451fdbd43b491fa701daac5de5e1c8" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" is_pre_election: true
I20240502 14:01:14.414431 9170 raft_consensus.cc:2454] T 50451fdbd43b491fa701daac5de5e1c8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 0.
W20240502 14:01:14.414937 9237 leader_election.cc:341] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459): Illegal state: must be running to vote when last-logged opid is not known
I20240502 14:01:14.415089 10306 tablet_bootstrap.cc:492] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de: No bootstrap required, opened a new log
I20240502 14:01:14.415165 10306 ts_tablet_manager.cc:1367] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 14:01:14.415272 10437 raft_consensus.cc:2781] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:14.415326 10437 raft_consensus.cc:482] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:14.415374 10437 raft_consensus.cc:3037] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.415737 10305 tablet_bootstrap.cc:492] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b: No bootstrap required, opened a new log
I20240502 14:01:14.415817 10305 ts_tablet_manager.cc:1367] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:01:14.415791 10306 raft_consensus.cc:348] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.415912 10306 raft_consensus.cc:374] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.415941 10306 raft_consensus.cc:729] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.416057 10306 consensus_queue.cc:260] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.416272 10305 raft_consensus.cc:348] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.416440 10305 raft_consensus.cc:374] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.416446 10306 ts_tablet_manager.cc:1398] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:14.416472 10305 raft_consensus.cc:729] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.416507 10437 raft_consensus.cc:504] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.416580 10305 consensus_queue.cc:260] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.416831 10305 ts_tablet_manager.cc:1398] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:14.417102 10437 leader_election.cc:288] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [CANDIDATE]: Term 1 election: Requested vote from peers 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679), bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837)
I20240502 14:01:14.417742 9829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50451fdbd43b491fa701daac5de5e1c8" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b"
I20240502 14:01:14.417851 9829 raft_consensus.cc:3037] T 50451fdbd43b491fa701daac5de5e1c8 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.418067 9565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50451fdbd43b491fa701daac5de5e1c8" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca"
I20240502 14:01:14.418177 9565 raft_consensus.cc:3037] T 50451fdbd43b491fa701daac5de5e1c8 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.418833 9829 raft_consensus.cc:2454] T 50451fdbd43b491fa701daac5de5e1c8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 1.
I20240502 14:01:14.419209 9565 raft_consensus.cc:2454] T 50451fdbd43b491fa701daac5de5e1c8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 1.
I20240502 14:01:14.419704 10304 tablet_bootstrap.cc:492] T 50451fdbd43b491fa701daac5de5e1c8 P bc9e4f2f668d48ea87d5aad4a2fb569d: No bootstrap required, opened a new log
I20240502 14:01:14.419788 10304 ts_tablet_manager.cc:1367] T 50451fdbd43b491fa701daac5de5e1c8 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240502 14:01:14.419826 9433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50451fdbd43b491fa701daac5de5e1c8" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de"
I20240502 14:01:14.419850 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50451fdbd43b491fa701daac5de5e1c8" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d"
I20240502 14:01:14.419932 9433 raft_consensus.cc:3037] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.420105 9170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50451fdbd43b491fa701daac5de5e1c8" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16"
I20240502 14:01:14.420209 9170 raft_consensus.cc:3037] T 50451fdbd43b491fa701daac5de5e1c8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.420207 10304 raft_consensus.cc:348] T 50451fdbd43b491fa701daac5de5e1c8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.420320 10304 raft_consensus.cc:374] T 50451fdbd43b491fa701daac5de5e1c8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.420385 10304 raft_consensus.cc:729] T 50451fdbd43b491fa701daac5de5e1c8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9e4f2f668d48ea87d5aad4a2fb569d, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.420477 9698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50451fdbd43b491fa701daac5de5e1c8" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288"
I20240502 14:01:14.420562 10300 tablet_bootstrap.cc:492] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16: No bootstrap required, opened a new log
I20240502 14:01:14.420547 10304 consensus_queue.cc:260] T 50451fdbd43b491fa701daac5de5e1c8 P bc9e4f2f668d48ea87d5aad4a2fb569d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.420599 9698 raft_consensus.cc:3037] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.420647 10300 ts_tablet_manager.cc:1367] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20240502 14:01:14.420869 10304 ts_tablet_manager.cc:1398] T 50451fdbd43b491fa701daac5de5e1c8 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:14.421056 10300 raft_consensus.cc:348] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.421162 10300 raft_consensus.cc:374] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.421191 10300 raft_consensus.cc:729] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83af1bc823a4f2ca531f0d6825f8e16, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.421231 9170 raft_consensus.cc:2454] T 50451fdbd43b491fa701daac5de5e1c8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 1.
I20240502 14:01:14.421598 9698 raft_consensus.cc:2454] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 1.
I20240502 14:01:14.421548 10300 consensus_queue.cc:260] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.421909 9038 raft_consensus.cc:3037] T 50451fdbd43b491fa701daac5de5e1c8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.421941 10300 ts_tablet_manager.cc:1398] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:14.422869 9038 raft_consensus.cc:2454] T 50451fdbd43b491fa701daac5de5e1c8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 1.
I20240502 14:01:14.422998 9433 raft_consensus.cc:2454] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 1.
I20240502 14:01:14.423269 9237 leader_election.cc:302] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0488b0e4ed994db19d40aed8c15f945b, bc9e4f2f668d48ea87d5aad4a2fb569d, ccfb5bbc1bb54793812d11170ac2d25f, d0f45862d7f14ce1976c1a37842f25ca; no voters:
I20240502 14:01:14.423585 9170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50451fdbd43b491fa701daac5de5e1c8" candidate_uuid: "efa836edab8d4853a7ca47a23117e288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" is_pre_election: true
I20240502 14:01:14.423761 9170 raft_consensus.cc:2379] T 50451fdbd43b491fa701daac5de5e1c8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa836edab8d4853a7ca47a23117e288 in current term 1: Already voted for candidate ccfb5bbc1bb54793812d11170ac2d25f in this term.
I20240502 14:01:14.424192 10437 raft_consensus.cc:2781] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:14.424607 10437 raft_consensus.cc:686] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Becoming Leader. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Running, Role: LEADER
I20240502 14:01:14.424762 10397 raft_consensus.cc:482] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:14.424796 10437 consensus_queue.cc:237] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.424865 10397 raft_consensus.cc:504] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.425563 10397 leader_election.cc:288] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:14.426410 9698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eef89d7e129492bab305a1b5009aace" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288" is_pre_election: true
I20240502 14:01:14.426550 9698 raft_consensus.cc:2454] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:14.426610 9564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eef89d7e129492bab305a1b5009aace" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca" is_pre_election: true
I20240502 14:01:14.426725 9564 raft_consensus.cc:2454] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:14.427028 9830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50451fdbd43b491fa701daac5de5e1c8" candidate_uuid: "efa836edab8d4853a7ca47a23117e288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b" is_pre_election: true
I20240502 14:01:14.427182 9830 raft_consensus.cc:2379] T 50451fdbd43b491fa701daac5de5e1c8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa836edab8d4853a7ca47a23117e288 in current term 1: Already voted for candidate ccfb5bbc1bb54793812d11170ac2d25f in this term.
I20240502 14:01:14.427595 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eef89d7e129492bab305a1b5009aace" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" is_pre_election: true
I20240502 14:01:14.427721 9038 raft_consensus.cc:2454] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:14.427867 9302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eef89d7e129492bab305a1b5009aace" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" is_pre_election: true
I20240502 14:01:14.427985 9302 raft_consensus.cc:2454] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:14.428020 9369 leader_election.cc:302] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1a490f170c614ee4900ef16c3287c7de, bc9e4f2f668d48ea87d5aad4a2fb569d, d0f45862d7f14ce1976c1a37842f25ca, efa836edab8d4853a7ca47a23117e288; no voters:
I20240502 14:01:14.428170 9565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50451fdbd43b491fa701daac5de5e1c8" candidate_uuid: "efa836edab8d4853a7ca47a23117e288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca" is_pre_election: true
I20240502 14:01:14.428308 9565 raft_consensus.cc:2379] T 50451fdbd43b491fa701daac5de5e1c8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa836edab8d4853a7ca47a23117e288 in current term 1: Already voted for candidate ccfb5bbc1bb54793812d11170ac2d25f in this term.
I20240502 14:01:14.428580 10397 raft_consensus.cc:2781] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:14.428650 10397 raft_consensus.cc:482] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:14.428709 10397 raft_consensus.cc:3037] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.428836 9170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eef89d7e129492bab305a1b5009aace" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" is_pre_election: true
I20240502 14:01:14.428948 9170 raft_consensus.cc:2454] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:14.429704 10397 raft_consensus.cc:504] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.430294 10397 leader_election.cc:288] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [CANDIDATE]: Term 1 election: Requested vote from peers bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:14.430408 9170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eef89d7e129492bab305a1b5009aace" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16"
I20240502 14:01:14.430500 9170 raft_consensus.cc:3037] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.430802 9830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eef89d7e129492bab305a1b5009aace" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b"
I20240502 14:01:14.430910 9830 raft_consensus.cc:3037] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.431324 9170 raft_consensus.cc:2454] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:14.431659 9302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eef89d7e129492bab305a1b5009aace" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f"
I20240502 14:01:14.431756 9302 raft_consensus.cc:3037] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.431901 9830 raft_consensus.cc:2454] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:14.431911 8890 catalog_manager.cc:5484] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: term changed from 0 to 1, leader changed from <none> to ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67). New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:14.432634 9302 raft_consensus.cc:2454] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:14.432809 9829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eef89d7e129492bab305a1b5009aace" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b" is_pre_election: true
I20240502 14:01:14.432838 9698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eef89d7e129492bab305a1b5009aace" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288"
I20240502 14:01:14.432924 9829 raft_consensus.cc:2362] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1. Re-sending same reply.
I20240502 14:01:14.432956 9698 raft_consensus.cc:3037] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.433676 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eef89d7e129492bab305a1b5009aace" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d"
I20240502 14:01:14.433809 9038 raft_consensus.cc:3037] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.433887 9698 raft_consensus.cc:2454] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:14.434116 9565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eef89d7e129492bab305a1b5009aace" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca"
I20240502 14:01:14.434238 9565 raft_consensus.cc:3037] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.434691 9038 raft_consensus.cc:2454] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:14.434909 9370 leader_election.cc:302] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0488b0e4ed994db19d40aed8c15f945b, 1a490f170c614ee4900ef16c3287c7de, d83af1bc823a4f2ca531f0d6825f8e16, efa836edab8d4853a7ca47a23117e288; no voters:
I20240502 14:01:14.435068 9565 raft_consensus.cc:2454] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:14.435478 10397 raft_consensus.cc:2781] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:14.435575 10397 raft_consensus.cc:686] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Becoming Leader. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Running, Role: LEADER
I20240502 14:01:14.435727 10397 consensus_queue.cc:237] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.437543 8890 catalog_manager.cc:5484] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a490f170c614ee4900ef16c3287c7de (127.24.66.68). New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:14.442008 9433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50451fdbd43b491fa701daac5de5e1c8" candidate_uuid: "efa836edab8d4853a7ca47a23117e288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de" is_pre_election: true
I20240502 14:01:14.442171 9433 raft_consensus.cc:2379] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa836edab8d4853a7ca47a23117e288 in current term 1: Already voted for candidate ccfb5bbc1bb54793812d11170ac2d25f in this term.
I20240502 14:01:14.442458 9633 leader_election.cc:302] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: ccfb5bbc1bb54793812d11170ac2d25f, efa836edab8d4853a7ca47a23117e288; no voters: 0488b0e4ed994db19d40aed8c15f945b, 1a490f170c614ee4900ef16c3287c7de, d0f45862d7f14ce1976c1a37842f25ca, d83af1bc823a4f2ca531f0d6825f8e16
I20240502 14:01:14.442750 10439 raft_consensus.cc:482] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:14.442847 10439 raft_consensus.cc:504] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.443292 10339 raft_consensus.cc:2726] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 14:01:14.443420 10439 leader_election.cc:288] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:14.443657 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50451fdbd43b491fa701daac5de5e1c8" candidate_uuid: "efa836edab8d4853a7ca47a23117e288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" is_pre_election: true
I20240502 14:01:14.443825 9038 raft_consensus.cc:2379] T 50451fdbd43b491fa701daac5de5e1c8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa836edab8d4853a7ca47a23117e288 in current term 1: Already voted for candidate ccfb5bbc1bb54793812d11170ac2d25f in this term.
I20240502 14:01:14.443818 9698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39bef244549d4bb7a980bf7bc52a01b4" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288" is_pre_election: true
I20240502 14:01:14.443948 9698 raft_consensus.cc:2454] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:14.444144 9037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39bef244549d4bb7a980bf7bc52a01b4" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" is_pre_election: true
I20240502 14:01:14.444141 9302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39bef244549d4bb7a980bf7bc52a01b4" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" is_pre_election: true
I20240502 14:01:14.444226 9433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39bef244549d4bb7a980bf7bc52a01b4" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de" is_pre_election: true
I20240502 14:01:14.444262 9037 raft_consensus.cc:2454] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:14.444267 9302 raft_consensus.cc:2454] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:14.444362 9433 raft_consensus.cc:2454] T 39bef244549d4bb7a980bf7bc52a01b4 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:14.444506 9170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39bef244549d4bb7a980bf7bc52a01b4" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" is_pre_election: true
I20240502 14:01:14.444588 9765 leader_election.cc:302] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0488b0e4ed994db19d40aed8c15f945b, 1a490f170c614ee4900ef16c3287c7de, bc9e4f2f668d48ea87d5aad4a2fb569d, efa836edab8d4853a7ca47a23117e288; no voters:
I20240502 14:01:14.444618 9170 raft_consensus.cc:2454] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:14.445055 10439 raft_consensus.cc:2781] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:14.445106 10439 raft_consensus.cc:482] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:14.445132 10439 raft_consensus.cc:3037] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.445119 9565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39bef244549d4bb7a980bf7bc52a01b4" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca" is_pre_election: true
I20240502 14:01:14.445223 9565 raft_consensus.cc:2454] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:14.445948 10439 raft_consensus.cc:504] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.446492 10439 leader_election.cc:288] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 election: Requested vote from peers 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:14.446624 9433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39bef244549d4bb7a980bf7bc52a01b4" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de"
I20240502 14:01:14.446627 9565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39bef244549d4bb7a980bf7bc52a01b4" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca"
I20240502 14:01:14.446722 9565 raft_consensus.cc:3037] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.446719 9433 raft_consensus.cc:3037] T 39bef244549d4bb7a980bf7bc52a01b4 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.446807 9302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39bef244549d4bb7a980bf7bc52a01b4" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f"
I20240502 14:01:14.446902 9302 raft_consensus.cc:3037] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.447211 9170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39bef244549d4bb7a980bf7bc52a01b4" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16"
I20240502 14:01:14.447327 9170 raft_consensus.cc:3037] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.447619 9565 raft_consensus.cc:2454] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:14.447687 9433 raft_consensus.cc:2454] T 39bef244549d4bb7a980bf7bc52a01b4 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:14.447821 9302 raft_consensus.cc:2454] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:14.448064 9698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39bef244549d4bb7a980bf7bc52a01b4" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288"
I20240502 14:01:14.448185 9698 raft_consensus.cc:3037] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.448289 9170 raft_consensus.cc:2454] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:14.448535 9037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39bef244549d4bb7a980bf7bc52a01b4" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d"
I20240502 14:01:14.448633 9037 raft_consensus.cc:3037] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.448568 9765 leader_election.cc:302] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0488b0e4ed994db19d40aed8c15f945b, 1a490f170c614ee4900ef16c3287c7de, ccfb5bbc1bb54793812d11170ac2d25f, d0f45862d7f14ce1976c1a37842f25ca; no voters:
I20240502 14:01:14.449194 9698 raft_consensus.cc:2454] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:14.449568 9037 raft_consensus.cc:2454] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:14.449990 10439 raft_consensus.cc:2781] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:14.450070 10439 raft_consensus.cc:686] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Becoming Leader. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Running, Role: LEADER
I20240502 14:01:14.450182 10439 consensus_queue.cc:237] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.452358 8890 catalog_manager.cc:5484] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71). New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } }
W20240502 14:01:14.572592 10498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:14.572832 10498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:14.575924 10498 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240502 14:01:14.654716 9302 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.654716 9170 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.655174 9698 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.655460 10439 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.655611 10524 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.655879 9433 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.656322 10359 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.656598 10359 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.660825 9037 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.661680 10321 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.662365 9169 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.663023 10437 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.664897 9434 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.665517 10437 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.665611 9038 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.666039 10318 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.666633 9829 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.669126 9565 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.669530 10439 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.669740 10318 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.671286 9698 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.672076 10318 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.679103 9565 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.679982 10318 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.689538 9830 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.690563 9036 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.690961 9168 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.691692 10397 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.691953 9565 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.692013 10397 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.692965 10398 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.693452 10313 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.693888 9302 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.697755 9698 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:14.698293 10399 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:14.698609 10399 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 204.625 ms
time per row: 0.102313 ms
I20240502 14:01:14.826102 8892 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46030:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240502 14:01:14.826723 8892 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_2 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240502 14:01:14.832847 9412 tablet_service.cc:1467] Processing CreateTablet for tablet 3c597826f8094c2e81bebfb87d4c77b5 (DEFAULT_TABLE table=rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.832835 9148 tablet_service.cc:1467] Processing CreateTablet for tablet 3c597826f8094c2e81bebfb87d4c77b5 (DEFAULT_TABLE table=rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.833163 9148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c597826f8094c2e81bebfb87d4c77b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.833163 9412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c597826f8094c2e81bebfb87d4c77b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.834359 9018 tablet_service.cc:1467] Processing CreateTablet for tablet 3c597826f8094c2e81bebfb87d4c77b5 (DEFAULT_TABLE table=rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.834640 9018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c597826f8094c2e81bebfb87d4c77b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.835613 9282 tablet_service.cc:1467] Processing CreateTablet for tablet 3c597826f8094c2e81bebfb87d4c77b5 (DEFAULT_TABLE table=rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.835915 9282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c597826f8094c2e81bebfb87d4c77b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.837098 9544 tablet_service.cc:1467] Processing CreateTablet for tablet 3c597826f8094c2e81bebfb87d4c77b5 (DEFAULT_TABLE table=rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.837378 9544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c597826f8094c2e81bebfb87d4c77b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.837631 9412 tablet_service.cc:1467] Processing CreateTablet for tablet f904eef49f9143cd957c8fa5c8ea25ea (DEFAULT_TABLE table=rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.838075 9412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f904eef49f9143cd957c8fa5c8ea25ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.838618 10306 tablet_bootstrap.cc:492] T 3c597826f8094c2e81bebfb87d4c77b5 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:14.839362 9545 tablet_service.cc:1467] Processing CreateTablet for tablet f904eef49f9143cd957c8fa5c8ea25ea (DEFAULT_TABLE table=rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.839506 10306 tablet_bootstrap.cc:654] T 3c597826f8094c2e81bebfb87d4c77b5 P 1a490f170c614ee4900ef16c3287c7de: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.839587 9545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f904eef49f9143cd957c8fa5c8ea25ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.840092 10300 tablet_bootstrap.cc:492] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16: Bootstrap starting.
I20240502 14:01:14.840484 10304 tablet_bootstrap.cc:492] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap starting.
I20240502 14:01:14.841527 10304 tablet_bootstrap.cc:654] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.841671 9546 tablet_service.cc:1467] Processing CreateTablet for tablet 6aed24ef77e54c8b8ff09e6709a886e4 (DEFAULT_TABLE table=rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.841890 9546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aed24ef77e54c8b8ff09e6709a886e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.842092 10301 tablet_bootstrap.cc:492] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f: Bootstrap starting.
I20240502 14:01:14.842573 10304 tablet_bootstrap.cc:492] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d: No bootstrap required, opened a new log
I20240502 14:01:14.842660 10304 ts_tablet_manager.cc:1367] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:14.842962 10301 tablet_bootstrap.cc:654] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.843168 10304 raft_consensus.cc:348] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.843305 10304 raft_consensus.cc:374] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.843339 10304 raft_consensus.cc:729] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9e4f2f668d48ea87d5aad4a2fb569d, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.843499 10304 consensus_queue.cc:260] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.843832 10304 ts_tablet_manager.cc:1398] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:14.843858 9413 tablet_service.cc:1467] Processing CreateTablet for tablet 6aed24ef77e54c8b8ff09e6709a886e4 (DEFAULT_TABLE table=rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.844087 9413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aed24ef77e54c8b8ff09e6709a886e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.844926 10303 tablet_bootstrap.cc:492] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap starting.
I20240502 14:01:14.845460 9677 tablet_service.cc:1467] Processing CreateTablet for tablet 3c597826f8094c2e81bebfb87d4c77b5 (DEFAULT_TABLE table=rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.845729 9677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c597826f8094c2e81bebfb87d4c77b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.845759 10303 tablet_bootstrap.cc:654] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.847656 9018 tablet_service.cc:1467] Processing CreateTablet for tablet f904eef49f9143cd957c8fa5c8ea25ea (DEFAULT_TABLE table=rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.847716 9017 tablet_service.cc:1467] Processing CreateTablet for tablet 6aed24ef77e54c8b8ff09e6709a886e4 (DEFAULT_TABLE table=rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.847944 10299 tablet_bootstrap.cc:492] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288: Bootstrap starting.
I20240502 14:01:14.847954 9017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aed24ef77e54c8b8ff09e6709a886e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.848143 9678 tablet_service.cc:1467] Processing CreateTablet for tablet f904eef49f9143cd957c8fa5c8ea25ea (DEFAULT_TABLE table=rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.848553 9676 tablet_service.cc:1467] Processing CreateTablet for tablet 6aed24ef77e54c8b8ff09e6709a886e4 (DEFAULT_TABLE table=rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.848762 9676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aed24ef77e54c8b8ff09e6709a886e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.848793 10299 tablet_bootstrap.cc:654] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.849329 9018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f904eef49f9143cd957c8fa5c8ea25ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.851524 10304 tablet_bootstrap.cc:492] T 6aed24ef77e54c8b8ff09e6709a886e4 P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap starting.
I20240502 14:01:14.851553 9280 tablet_service.cc:1467] Processing CreateTablet for tablet 6aed24ef77e54c8b8ff09e6709a886e4 (DEFAULT_TABLE table=rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.851691 9150 tablet_service.cc:1467] Processing CreateTablet for tablet 6aed24ef77e54c8b8ff09e6709a886e4 (DEFAULT_TABLE table=rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.851809 9280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aed24ef77e54c8b8ff09e6709a886e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.851963 9150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aed24ef77e54c8b8ff09e6709a886e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.852456 10304 tablet_bootstrap.cc:654] T 6aed24ef77e54c8b8ff09e6709a886e4 P bc9e4f2f668d48ea87d5aad4a2fb569d: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.854437 10299 tablet_bootstrap.cc:492] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288: No bootstrap required, opened a new log
I20240502 14:01:14.854552 10299 ts_tablet_manager.cc:1367] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 14:01:14.855053 10299 raft_consensus.cc:348] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.855185 10299 raft_consensus.cc:374] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.855216 10299 raft_consensus.cc:729] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa836edab8d4853a7ca47a23117e288, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.855257 9809 tablet_service.cc:1467] Processing CreateTablet for tablet 3c597826f8094c2e81bebfb87d4c77b5 (DEFAULT_TABLE table=rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.855358 10299 consensus_queue.cc:260] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.855553 9809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c597826f8094c2e81bebfb87d4c77b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.855661 10299 ts_tablet_manager.cc:1398] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:14.855845 10299 tablet_bootstrap.cc:492] T 6aed24ef77e54c8b8ff09e6709a886e4 P efa836edab8d4853a7ca47a23117e288: Bootstrap starting.
I20240502 14:01:14.856727 10299 tablet_bootstrap.cc:654] T 6aed24ef77e54c8b8ff09e6709a886e4 P efa836edab8d4853a7ca47a23117e288: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.856678 9810 tablet_service.cc:1467] Processing CreateTablet for tablet 6aed24ef77e54c8b8ff09e6709a886e4 (DEFAULT_TABLE table=rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.856914 9810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aed24ef77e54c8b8ff09e6709a886e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.857836 10305 tablet_bootstrap.cc:492] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b: Bootstrap starting.
I20240502 14:01:14.858486 10299 tablet_bootstrap.cc:492] T 6aed24ef77e54c8b8ff09e6709a886e4 P efa836edab8d4853a7ca47a23117e288: No bootstrap required, opened a new log
I20240502 14:01:14.858568 10299 ts_tablet_manager.cc:1367] T 6aed24ef77e54c8b8ff09e6709a886e4 P efa836edab8d4853a7ca47a23117e288: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:01:14.858659 10305 tablet_bootstrap.cc:654] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.858848 9808 tablet_service.cc:1467] Processing CreateTablet for tablet f904eef49f9143cd957c8fa5c8ea25ea (DEFAULT_TABLE table=rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.859043 9808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f904eef49f9143cd957c8fa5c8ea25ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.859189 10299 raft_consensus.cc:348] T 6aed24ef77e54c8b8ff09e6709a886e4 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.859287 10299 raft_consensus.cc:374] T 6aed24ef77e54c8b8ff09e6709a886e4 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.859308 10299 raft_consensus.cc:729] T 6aed24ef77e54c8b8ff09e6709a886e4 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa836edab8d4853a7ca47a23117e288, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.859477 10299 consensus_queue.cc:260] T 6aed24ef77e54c8b8ff09e6709a886e4 P efa836edab8d4853a7ca47a23117e288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.859743 10299 ts_tablet_manager.cc:1398] T 6aed24ef77e54c8b8ff09e6709a886e4 P efa836edab8d4853a7ca47a23117e288: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:14.860456 10305 tablet_bootstrap.cc:492] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b: No bootstrap required, opened a new log
I20240502 14:01:14.860550 10305 ts_tablet_manager.cc:1367] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:01:14.860898 10305 raft_consensus.cc:348] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.861043 10305 raft_consensus.cc:374] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.861080 10305 raft_consensus.cc:729] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.861196 10305 consensus_queue.cc:260] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.861455 10305 ts_tablet_manager.cc:1398] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:14.861521 10300 tablet_bootstrap.cc:654] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.861624 10305 tablet_bootstrap.cc:492] T 6aed24ef77e54c8b8ff09e6709a886e4 P 0488b0e4ed994db19d40aed8c15f945b: Bootstrap starting.
I20240502 14:01:14.862466 10305 tablet_bootstrap.cc:654] T 6aed24ef77e54c8b8ff09e6709a886e4 P 0488b0e4ed994db19d40aed8c15f945b: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.862530 9678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f904eef49f9143cd957c8fa5c8ea25ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.863832 10305 tablet_bootstrap.cc:492] T 6aed24ef77e54c8b8ff09e6709a886e4 P 0488b0e4ed994db19d40aed8c15f945b: No bootstrap required, opened a new log
I20240502 14:01:14.863912 10305 ts_tablet_manager.cc:1367] T 6aed24ef77e54c8b8ff09e6709a886e4 P 0488b0e4ed994db19d40aed8c15f945b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:14.864571 10305 raft_consensus.cc:348] T 6aed24ef77e54c8b8ff09e6709a886e4 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.864707 10305 raft_consensus.cc:374] T 6aed24ef77e54c8b8ff09e6709a886e4 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.864742 10305 raft_consensus.cc:729] T 6aed24ef77e54c8b8ff09e6709a886e4 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.864887 10305 consensus_queue.cc:260] T 6aed24ef77e54c8b8ff09e6709a886e4 P 0488b0e4ed994db19d40aed8c15f945b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.865458 10305 ts_tablet_manager.cc:1398] T 6aed24ef77e54c8b8ff09e6709a886e4 P 0488b0e4ed994db19d40aed8c15f945b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:01:14.865643 10305 tablet_bootstrap.cc:492] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b: Bootstrap starting.
I20240502 14:01:14.866537 10305 tablet_bootstrap.cc:654] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.866750 10299 tablet_bootstrap.cc:492] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288: Bootstrap starting.
I20240502 14:01:14.867660 10301 tablet_bootstrap.cc:492] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f: No bootstrap required, opened a new log
I20240502 14:01:14.867762 10301 ts_tablet_manager.cc:1367] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent bootstrapping tablet: real 0.026s user 0.000s sys 0.001s
I20240502 14:01:14.867779 10299 tablet_bootstrap.cc:654] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.868243 10301 raft_consensus.cc:348] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.868388 10301 raft_consensus.cc:374] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.868429 10301 raft_consensus.cc:729] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.868606 10301 consensus_queue.cc:260] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.868927 10301 ts_tablet_manager.cc:1398] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:14.869103 10301 tablet_bootstrap.cc:492] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f: Bootstrap starting.
I20240502 14:01:14.869128 9282 tablet_service.cc:1467] Processing CreateTablet for tablet f904eef49f9143cd957c8fa5c8ea25ea (DEFAULT_TABLE table=rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.869568 9148 tablet_service.cc:1467] Processing CreateTablet for tablet f904eef49f9143cd957c8fa5c8ea25ea (DEFAULT_TABLE table=rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:14.869861 9148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f904eef49f9143cd957c8fa5c8ea25ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.869935 10301 tablet_bootstrap.cc:654] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.870180 10300 tablet_bootstrap.cc:492] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16: No bootstrap required, opened a new log
I20240502 14:01:14.870265 10300 ts_tablet_manager.cc:1367] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.000s
I20240502 14:01:14.870566 9282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f904eef49f9143cd957c8fa5c8ea25ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:14.870745 10300 raft_consensus.cc:348] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.870872 10300 raft_consensus.cc:374] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.870905 10300 raft_consensus.cc:729] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83af1bc823a4f2ca531f0d6825f8e16, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.871050 10300 consensus_queue.cc:260] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.871381 10300 ts_tablet_manager.cc:1398] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:14.871569 10300 tablet_bootstrap.cc:492] T 6aed24ef77e54c8b8ff09e6709a886e4 P d83af1bc823a4f2ca531f0d6825f8e16: Bootstrap starting.
I20240502 14:01:14.872712 10300 tablet_bootstrap.cc:654] T 6aed24ef77e54c8b8ff09e6709a886e4 P d83af1bc823a4f2ca531f0d6825f8e16: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.873320 10303 tablet_bootstrap.cc:492] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca: No bootstrap required, opened a new log
I20240502 14:01:14.873416 10303 ts_tablet_manager.cc:1367] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.000s
I20240502 14:01:14.873497 10306 tablet_bootstrap.cc:492] T 3c597826f8094c2e81bebfb87d4c77b5 P 1a490f170c614ee4900ef16c3287c7de: No bootstrap required, opened a new log
I20240502 14:01:14.873618 10306 ts_tablet_manager.cc:1367] T 3c597826f8094c2e81bebfb87d4c77b5 P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.035s user 0.001s sys 0.000s
I20240502 14:01:14.873693 10299 tablet_bootstrap.cc:492] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288: No bootstrap required, opened a new log
I20240502 14:01:14.873778 10299 ts_tablet_manager.cc:1367] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20240502 14:01:14.873924 10303 raft_consensus.cc:348] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.874043 10303 raft_consensus.cc:374] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.874078 10303 raft_consensus.cc:729] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.874157 10306 raft_consensus.cc:348] T 3c597826f8094c2e81bebfb87d4c77b5 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.874199 10303 consensus_queue.cc:260] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.874308 10306 raft_consensus.cc:374] T 3c597826f8094c2e81bebfb87d4c77b5 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.874239 10299 raft_consensus.cc:348] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.874357 10306 raft_consensus.cc:729] T 3c597826f8094c2e81bebfb87d4c77b5 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.874367 10299 raft_consensus.cc:374] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.874400 10299 raft_consensus.cc:729] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa836edab8d4853a7ca47a23117e288, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.874516 10303 ts_tablet_manager.cc:1398] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:14.874563 10300 tablet_bootstrap.cc:492] T 6aed24ef77e54c8b8ff09e6709a886e4 P d83af1bc823a4f2ca531f0d6825f8e16: No bootstrap required, opened a new log
I20240502 14:01:14.874504 10306 consensus_queue.cc:260] T 3c597826f8094c2e81bebfb87d4c77b5 P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.874531 10299 consensus_queue.cc:260] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.874644 10300 ts_tablet_manager.cc:1367] T 6aed24ef77e54c8b8ff09e6709a886e4 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 14:01:14.874691 10303 tablet_bootstrap.cc:492] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap starting.
I20240502 14:01:14.874794 10299 ts_tablet_manager.cc:1398] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:14.874804 10306 ts_tablet_manager.cc:1398] T 3c597826f8094c2e81bebfb87d4c77b5 P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:01:14.874979 10306 tablet_bootstrap.cc:492] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:14.875113 10300 raft_consensus.cc:348] T 6aed24ef77e54c8b8ff09e6709a886e4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.875216 10300 raft_consensus.cc:374] T 6aed24ef77e54c8b8ff09e6709a886e4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.875247 10300 raft_consensus.cc:729] T 6aed24ef77e54c8b8ff09e6709a886e4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83af1bc823a4f2ca531f0d6825f8e16, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.875394 10300 consensus_queue.cc:260] T 6aed24ef77e54c8b8ff09e6709a886e4 P d83af1bc823a4f2ca531f0d6825f8e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.875583 10303 tablet_bootstrap.cc:654] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.875687 10300 ts_tablet_manager.cc:1398] T 6aed24ef77e54c8b8ff09e6709a886e4 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:14.875864 10300 tablet_bootstrap.cc:492] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16: Bootstrap starting.
I20240502 14:01:14.875900 10306 tablet_bootstrap.cc:654] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.876880 10300 tablet_bootstrap.cc:654] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.877043 10303 tablet_bootstrap.cc:492] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca: No bootstrap required, opened a new log
I20240502 14:01:14.877113 10303 ts_tablet_manager.cc:1367] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:14.877573 10303 raft_consensus.cc:348] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.877691 10303 raft_consensus.cc:374] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.877730 10303 raft_consensus.cc:729] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.877887 10305 tablet_bootstrap.cc:492] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b: No bootstrap required, opened a new log
I20240502 14:01:14.877897 10303 consensus_queue.cc:260] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.877993 10305 ts_tablet_manager.cc:1367] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 14:01:14.878183 10314 raft_consensus.cc:482] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:14.878254 10303 ts_tablet_manager.cc:1398] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:14.878270 10314 raft_consensus.cc:504] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.878412 10303 tablet_bootstrap.cc:492] T 6aed24ef77e54c8b8ff09e6709a886e4 P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap starting.
I20240502 14:01:14.878608 10305 raft_consensus.cc:348] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.878728 10305 raft_consensus.cc:374] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.878762 10305 raft_consensus.cc:729] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.878899 10305 consensus_queue.cc:260] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.879181 10305 ts_tablet_manager.cc:1398] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:14.879540 10303 tablet_bootstrap.cc:654] T 6aed24ef77e54c8b8ff09e6709a886e4 P d0f45862d7f14ce1976c1a37842f25ca: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.880107 10314 leader_election.cc:288] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749), 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:14.881222 10523 raft_consensus.cc:482] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:14.881331 10523 raft_consensus.cc:504] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.882007 10523 leader_election.cc:288] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:14.882665 10304 tablet_bootstrap.cc:492] T 6aed24ef77e54c8b8ff09e6709a886e4 P bc9e4f2f668d48ea87d5aad4a2fb569d: No bootstrap required, opened a new log
I20240502 14:01:14.882776 10304 ts_tablet_manager.cc:1367] T 6aed24ef77e54c8b8ff09e6709a886e4 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent bootstrapping tablet: real 0.031s user 0.000s sys 0.002s
I20240502 14:01:14.882761 9698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f904eef49f9143cd957c8fa5c8ea25ea" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288" is_pre_election: true
I20240502 14:01:14.882894 9698 raft_consensus.cc:2454] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:14.883138 9302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f904eef49f9143cd957c8fa5c8ea25ea" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" is_pre_election: true
I20240502 14:01:14.883280 10304 raft_consensus.cc:348] T 6aed24ef77e54c8b8ff09e6709a886e4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.883409 10304 raft_consensus.cc:374] T 6aed24ef77e54c8b8ff09e6709a886e4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.883442 10304 raft_consensus.cc:729] T 6aed24ef77e54c8b8ff09e6709a886e4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9e4f2f668d48ea87d5aad4a2fb569d, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.883569 10304 consensus_queue.cc:260] T 6aed24ef77e54c8b8ff09e6709a886e4 P bc9e4f2f668d48ea87d5aad4a2fb569d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.883886 10304 ts_tablet_manager.cc:1398] T 6aed24ef77e54c8b8ff09e6709a886e4 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:14.883900 9564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f904eef49f9143cd957c8fa5c8ea25ea" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca" is_pre_election: true
I20240502 14:01:14.884011 9564 raft_consensus.cc:2454] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:14.884056 10304 tablet_bootstrap.cc:492] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap starting.
W20240502 14:01:14.884697 9763 leader_election.cc:341] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749): Illegal state: must be running to vote when last-logged opid is not known
I20240502 14:01:14.885087 10304 tablet_bootstrap.cc:654] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.885380 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f904eef49f9143cd957c8fa5c8ea25ea" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" is_pre_election: true
I20240502 14:01:14.885913 9433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f904eef49f9143cd957c8fa5c8ea25ea" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de" is_pre_election: true
I20240502 14:01:14.886884 9169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f904eef49f9143cd957c8fa5c8ea25ea" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" is_pre_election: true
I20240502 14:01:14.887102 10301 tablet_bootstrap.cc:492] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f: No bootstrap required, opened a new log
I20240502 14:01:14.887197 10301 ts_tablet_manager.cc:1367] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.000s
I20240502 14:01:14.887696 10301 raft_consensus.cc:348] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.887809 10301 raft_consensus.cc:374] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.887837 10301 raft_consensus.cc:729] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.887951 10301 consensus_queue.cc:260] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.888005 9564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c597826f8094c2e81bebfb87d4c77b5" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca" is_pre_election: true
I20240502 14:01:14.888129 9564 raft_consensus.cc:2454] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 0.
I20240502 14:01:14.888204 10301 ts_tablet_manager.cc:1398] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:14.888433 10301 tablet_bootstrap.cc:492] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f: Bootstrap starting.
W20240502 14:01:14.888407 9765 leader_election.cc:341] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837): Illegal state: must be running to vote when last-logged opid is not known
W20240502 14:01:14.888547 9765 leader_election.cc:341] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907): Illegal state: must be running to vote when last-logged opid is not known
W20240502 14:01:14.888624 9765 leader_election.cc:341] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459): Illegal state: must be running to vote when last-logged opid is not known
I20240502 14:01:14.888659 9765 leader_election.cc:302] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 0488b0e4ed994db19d40aed8c15f945b, d0f45862d7f14ce1976c1a37842f25ca, efa836edab8d4853a7ca47a23117e288; no voters: 1a490f170c614ee4900ef16c3287c7de, bc9e4f2f668d48ea87d5aad4a2fb569d, ccfb5bbc1bb54793812d11170ac2d25f, d83af1bc823a4f2ca531f0d6825f8e16
I20240502 14:01:14.888931 10525 raft_consensus.cc:2726] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 14:01:14.889403 10301 tablet_bootstrap.cc:654] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.890739 9698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c597826f8094c2e81bebfb87d4c77b5" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288" is_pre_election: true
I20240502 14:01:14.890879 9698 raft_consensus.cc:2454] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 0.
I20240502 14:01:14.892689 10301 tablet_bootstrap.cc:492] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f: No bootstrap required, opened a new log
I20240502 14:01:14.892791 10301 ts_tablet_manager.cc:1367] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:01:14.893304 10301 raft_consensus.cc:348] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.893443 10301 raft_consensus.cc:374] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.893481 10301 raft_consensus.cc:729] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.893642 10301 consensus_queue.cc:260] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.893936 10301 ts_tablet_manager.cc:1398] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:14.893956 10300 tablet_bootstrap.cc:492] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16: No bootstrap required, opened a new log
I20240502 14:01:14.894040 10300 ts_tablet_manager.cc:1367] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240502 14:01:14.894479 10300 raft_consensus.cc:348] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.894613 10300 raft_consensus.cc:374] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.894649 10300 raft_consensus.cc:729] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83af1bc823a4f2ca531f0d6825f8e16, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.894753 10300 consensus_queue.cc:260] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.895030 10300 ts_tablet_manager.cc:1398] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:14.895622 10306 tablet_bootstrap.cc:492] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de: No bootstrap required, opened a new log
I20240502 14:01:14.895756 10306 ts_tablet_manager.cc:1367] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20240502 14:01:14.896219 10306 raft_consensus.cc:348] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.896404 10306 raft_consensus.cc:374] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.896448 10306 raft_consensus.cc:729] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.896579 10306 consensus_queue.cc:260] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.896865 10306 ts_tablet_manager.cc:1398] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:14.897022 10306 tablet_bootstrap.cc:492] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:14.897154 10303 tablet_bootstrap.cc:492] T 6aed24ef77e54c8b8ff09e6709a886e4 P d0f45862d7f14ce1976c1a37842f25ca: No bootstrap required, opened a new log
I20240502 14:01:14.897284 10303 ts_tablet_manager.cc:1367] T 6aed24ef77e54c8b8ff09e6709a886e4 P d0f45862d7f14ce1976c1a37842f25ca: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20240502 14:01:14.897740 10303 raft_consensus.cc:348] T 6aed24ef77e54c8b8ff09e6709a886e4 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.897868 10303 raft_consensus.cc:374] T 6aed24ef77e54c8b8ff09e6709a886e4 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.897907 10303 raft_consensus.cc:729] T 6aed24ef77e54c8b8ff09e6709a886e4 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.898020 10306 tablet_bootstrap.cc:654] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:14.898056 10303 consensus_queue.cc:260] T 6aed24ef77e54c8b8ff09e6709a886e4 P d0f45862d7f14ce1976c1a37842f25ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.898448 10303 ts_tablet_manager.cc:1398] T 6aed24ef77e54c8b8ff09e6709a886e4 P d0f45862d7f14ce1976c1a37842f25ca: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:01:14.899806 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c597826f8094c2e81bebfb87d4c77b5" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" is_pre_election: true
I20240502 14:01:14.899961 9036 raft_consensus.cc:2454] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 0.
I20240502 14:01:14.900410 10304 tablet_bootstrap.cc:492] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d: No bootstrap required, opened a new log
I20240502 14:01:14.900504 10304 ts_tablet_manager.cc:1367] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 14:01:14.900631 9105 leader_election.cc:302] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: bc9e4f2f668d48ea87d5aad4a2fb569d, d0f45862d7f14ce1976c1a37842f25ca, d83af1bc823a4f2ca531f0d6825f8e16, efa836edab8d4853a7ca47a23117e288; no voters:
I20240502 14:01:14.900902 10314 raft_consensus.cc:2781] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:14.900971 10314 raft_consensus.cc:482] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:14.901012 10314 raft_consensus.cc:3037] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.900955 10304 raft_consensus.cc:348] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.901082 10304 raft_consensus.cc:374] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.901114 10304 raft_consensus.cc:729] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9e4f2f668d48ea87d5aad4a2fb569d, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.901273 10304 consensus_queue.cc:260] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.901567 10304 ts_tablet_manager.cc:1398] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:14.901803 10306 tablet_bootstrap.cc:492] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de: No bootstrap required, opened a new log
I20240502 14:01:14.901885 10306 ts_tablet_manager.cc:1367] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:01:14.902081 10314 raft_consensus.cc:504] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.902215 9433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c597826f8094c2e81bebfb87d4c77b5" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de" is_pre_election: true
I20240502 14:01:14.902325 9433 raft_consensus.cc:2454] T 3c597826f8094c2e81bebfb87d4c77b5 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 0.
I20240502 14:01:14.902313 10306 raft_consensus.cc:348] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.902472 10306 raft_consensus.cc:374] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:14.902511 10306 raft_consensus.cc:729] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: FOLLOWER
I20240502 14:01:14.902642 10306 consensus_queue.cc:260] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.902742 10314 leader_election.cc:288] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [CANDIDATE]: Term 1 election: Requested vote from peers bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749), 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:14.902967 10306 ts_tablet_manager.cc:1398] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:14.903021 9564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c597826f8094c2e81bebfb87d4c77b5" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca"
I20240502 14:01:14.903131 9564 raft_consensus.cc:3037] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.903292 9698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c597826f8094c2e81bebfb87d4c77b5" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288"
I20240502 14:01:14.903409 9698 raft_consensus.cc:3037] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.903920 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c597826f8094c2e81bebfb87d4c77b5" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d"
I20240502 14:01:14.904023 9036 raft_consensus.cc:3037] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.904134 9564 raft_consensus.cc:2454] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 1.
I20240502 14:01:14.904461 9698 raft_consensus.cc:2454] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 1.
I20240502 14:01:14.905121 9036 raft_consensus.cc:2454] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 1.
I20240502 14:01:14.905563 9302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c597826f8094c2e81bebfb87d4c77b5" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" is_pre_election: true
I20240502 14:01:14.905647 9301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c597826f8094c2e81bebfb87d4c77b5" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f"
I20240502 14:01:14.905678 9830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c597826f8094c2e81bebfb87d4c77b5" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b"
I20240502 14:01:14.905707 9302 raft_consensus.cc:2454] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 0.
I20240502 14:01:14.905792 9830 raft_consensus.cc:3037] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.906069 9105 leader_election.cc:302] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: bc9e4f2f668d48ea87d5aad4a2fb569d, d0f45862d7f14ce1976c1a37842f25ca, d83af1bc823a4f2ca531f0d6825f8e16, efa836edab8d4853a7ca47a23117e288; no voters:
I20240502 14:01:14.906513 9829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c597826f8094c2e81bebfb87d4c77b5" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b" is_pre_election: true
I20240502 14:01:14.906810 9830 raft_consensus.cc:2454] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 1.
I20240502 14:01:14.907039 9433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c597826f8094c2e81bebfb87d4c77b5" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de"
I20240502 14:01:14.907147 9433 raft_consensus.cc:3037] T 3c597826f8094c2e81bebfb87d4c77b5 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.907523 10314 raft_consensus.cc:2781] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:14.907711 10314 raft_consensus.cc:686] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Becoming Leader. State: Replica: d83af1bc823a4f2ca531f0d6825f8e16, State: Running, Role: LEADER
I20240502 14:01:14.908031 9433 raft_consensus.cc:2454] T 3c597826f8094c2e81bebfb87d4c77b5 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 1.
I20240502 14:01:14.908054 10314 consensus_queue.cc:237] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:14.911016 8892 catalog_manager.cc:5484] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: term changed from 0 to 1, leader changed from <none> to d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66). New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:14.922192 10427 raft_consensus.cc:482] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:14.922371 10427 raft_consensus.cc:504] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.924058 10427 leader_election.cc:288] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:14.928885 9433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f904eef49f9143cd957c8fa5c8ea25ea" candidate_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de" is_pre_election: true
I20240502 14:01:14.929081 9433 raft_consensus.cc:2454] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc9e4f2f668d48ea87d5aad4a2fb569d in term 0.
I20240502 14:01:14.931561 9698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f904eef49f9143cd957c8fa5c8ea25ea" candidate_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288" is_pre_election: true
I20240502 14:01:14.931744 9698 raft_consensus.cc:2454] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc9e4f2f668d48ea87d5aad4a2fb569d in term 0.
I20240502 14:01:14.931722 9301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f904eef49f9143cd957c8fa5c8ea25ea" candidate_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" is_pre_election: true
I20240502 14:01:14.931869 9301 raft_consensus.cc:2454] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc9e4f2f668d48ea87d5aad4a2fb569d in term 0.
I20240502 14:01:14.932113 9830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f904eef49f9143cd957c8fa5c8ea25ea" candidate_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b" is_pre_election: true
I20240502 14:01:14.932250 9830 raft_consensus.cc:2454] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc9e4f2f668d48ea87d5aad4a2fb569d in term 0.
I20240502 14:01:14.932574 8971 leader_election.cc:302] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1a490f170c614ee4900ef16c3287c7de, bc9e4f2f668d48ea87d5aad4a2fb569d, ccfb5bbc1bb54793812d11170ac2d25f, efa836edab8d4853a7ca47a23117e288; no voters:
I20240502 14:01:14.932685 9564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f904eef49f9143cd957c8fa5c8ea25ea" candidate_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca" is_pre_election: true
I20240502 14:01:14.932827 9564 raft_consensus.cc:2454] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc9e4f2f668d48ea87d5aad4a2fb569d in term 0.
I20240502 14:01:14.932861 10427 raft_consensus.cc:2781] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:14.932929 10427 raft_consensus.cc:482] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:14.932981 10427 raft_consensus.cc:3037] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.933990 10427 raft_consensus.cc:504] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.934321 10397 raft_consensus.cc:482] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:14.934417 9169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f904eef49f9143cd957c8fa5c8ea25ea" candidate_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" is_pre_election: true
I20240502 14:01:14.934435 10397 raft_consensus.cc:504] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.934573 9169 raft_consensus.cc:2454] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc9e4f2f668d48ea87d5aad4a2fb569d in term 0.
I20240502 14:01:14.934654 10427 leader_election.cc:288] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [CANDIDATE]: Term 1 election: Requested vote from peers 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:14.935104 9433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f904eef49f9143cd957c8fa5c8ea25ea" candidate_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de"
I20240502 14:01:14.935209 10397 leader_election.cc:288] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679), bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:14.935518 9830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aed24ef77e54c8b8ff09e6709a886e4" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b" is_pre_election: true
I20240502 14:01:14.935657 9830 raft_consensus.cc:2454] T 6aed24ef77e54c8b8ff09e6709a886e4 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:14.935684 9565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aed24ef77e54c8b8ff09e6709a886e4" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca" is_pre_election: true
I20240502 14:01:14.935683 9696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aed24ef77e54c8b8ff09e6709a886e4" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288" is_pre_election: true
I20240502 14:01:14.935817 9565 raft_consensus.cc:2454] T 6aed24ef77e54c8b8ff09e6709a886e4 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:14.935822 9696 raft_consensus.cc:2454] T 6aed24ef77e54c8b8ff09e6709a886e4 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:14.935994 9301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aed24ef77e54c8b8ff09e6709a886e4" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" is_pre_election: true
I20240502 14:01:14.936128 9301 raft_consensus.cc:2454] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:14.935215 9433 raft_consensus.cc:3037] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.936507 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aed24ef77e54c8b8ff09e6709a886e4" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" is_pre_election: true
I20240502 14:01:14.936501 9367 leader_election.cc:302] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0488b0e4ed994db19d40aed8c15f945b, 1a490f170c614ee4900ef16c3287c7de, ccfb5bbc1bb54793812d11170ac2d25f, d0f45862d7f14ce1976c1a37842f25ca; no voters:
I20240502 14:01:14.936641 9036 raft_consensus.cc:2454] T 6aed24ef77e54c8b8ff09e6709a886e4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:14.936702 10397 raft_consensus.cc:2781] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:14.936754 10397 raft_consensus.cc:482] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:14.936795 10397 raft_consensus.cc:3037] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.937052 9564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f904eef49f9143cd957c8fa5c8ea25ea" candidate_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca"
I20240502 14:01:14.937162 9564 raft_consensus.cc:3037] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.937489 9433 raft_consensus.cc:2454] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc9e4f2f668d48ea87d5aad4a2fb569d in term 1.
I20240502 14:01:14.937785 9168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aed24ef77e54c8b8ff09e6709a886e4" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" is_pre_election: true
I20240502 14:01:14.937774 10397 raft_consensus.cc:504] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.937916 9168 raft_consensus.cc:2454] T 6aed24ef77e54c8b8ff09e6709a886e4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:14.938153 9564 raft_consensus.cc:2454] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc9e4f2f668d48ea87d5aad4a2fb569d in term 1.
I20240502 14:01:14.938220 9169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f904eef49f9143cd957c8fa5c8ea25ea" candidate_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16"
I20240502 14:01:14.938336 9169 raft_consensus.cc:3037] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.938385 10397 leader_election.cc:288] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [CANDIDATE]: Term 1 election: Requested vote from peers d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679), bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:14.938659 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aed24ef77e54c8b8ff09e6709a886e4" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d"
I20240502 14:01:14.938777 9036 raft_consensus.cc:3037] T 6aed24ef77e54c8b8ff09e6709a886e4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.939376 9169 raft_consensus.cc:2454] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc9e4f2f668d48ea87d5aad4a2fb569d in term 1.
I20240502 14:01:14.939560 9036 raft_consensus.cc:2454] T 6aed24ef77e54c8b8ff09e6709a886e4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:14.939741 8973 leader_election.cc:302] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1a490f170c614ee4900ef16c3287c7de, bc9e4f2f668d48ea87d5aad4a2fb569d, d0f45862d7f14ce1976c1a37842f25ca, d83af1bc823a4f2ca531f0d6825f8e16; no voters:
I20240502 14:01:14.940033 9168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aed24ef77e54c8b8ff09e6709a886e4" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16"
I20240502 14:01:14.940035 9696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aed24ef77e54c8b8ff09e6709a886e4" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288"
I20240502 14:01:14.940157 9168 raft_consensus.cc:3037] T 6aed24ef77e54c8b8ff09e6709a886e4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.940158 9696 raft_consensus.cc:3037] T 6aed24ef77e54c8b8ff09e6709a886e4 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.940255 9829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f904eef49f9143cd957c8fa5c8ea25ea" candidate_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b"
I20240502 14:01:14.940320 9302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f904eef49f9143cd957c8fa5c8ea25ea" candidate_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f"
I20240502 14:01:14.940420 9829 raft_consensus.cc:3037] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.940491 9302 raft_consensus.cc:3037] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.941147 9168 raft_consensus.cc:2454] T 6aed24ef77e54c8b8ff09e6709a886e4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:14.941148 9696 raft_consensus.cc:2454] T 6aed24ef77e54c8b8ff09e6709a886e4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:14.941505 9829 raft_consensus.cc:2454] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc9e4f2f668d48ea87d5aad4a2fb569d in term 1.
I20240502 14:01:14.941632 9698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f904eef49f9143cd957c8fa5c8ea25ea" candidate_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288"
I20240502 14:01:14.941727 9698 raft_consensus.cc:3037] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.941906 9302 raft_consensus.cc:2454] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc9e4f2f668d48ea87d5aad4a2fb569d in term 1.
I20240502 14:01:14.942044 9369 leader_election.cc:302] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1a490f170c614ee4900ef16c3287c7de, bc9e4f2f668d48ea87d5aad4a2fb569d, d83af1bc823a4f2ca531f0d6825f8e16, efa836edab8d4853a7ca47a23117e288; no voters:
I20240502 14:01:14.942157 9830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aed24ef77e54c8b8ff09e6709a886e4" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b"
I20240502 14:01:14.942262 9830 raft_consensus.cc:3037] T 6aed24ef77e54c8b8ff09e6709a886e4 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.942320 10397 raft_consensus.cc:2781] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:14.942420 10397 raft_consensus.cc:686] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Becoming Leader. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Running, Role: LEADER
I20240502 14:01:14.942644 9698 raft_consensus.cc:2454] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc9e4f2f668d48ea87d5aad4a2fb569d in term 1.
I20240502 14:01:14.942610 10397 consensus_queue.cc:237] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.943195 9830 raft_consensus.cc:2454] T 6aed24ef77e54c8b8ff09e6709a886e4 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:14.943703 9301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aed24ef77e54c8b8ff09e6709a886e4" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f"
I20240502 14:01:14.943826 9301 raft_consensus.cc:3037] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.944751 10427 raft_consensus.cc:2781] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:14.944885 9301 raft_consensus.cc:2454] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:14.944978 10427 raft_consensus.cc:686] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 LEADER]: Becoming Leader. State: Replica: bc9e4f2f668d48ea87d5aad4a2fb569d, State: Running, Role: LEADER
I20240502 14:01:14.945060 8892 catalog_manager.cc:5484] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a490f170c614ee4900ef16c3287c7de (127.24.66.68). New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:14.945263 9564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aed24ef77e54c8b8ff09e6709a886e4" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca"
I20240502 14:01:14.945379 9564 raft_consensus.cc:3037] T 6aed24ef77e54c8b8ff09e6709a886e4 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:14.945365 10427 consensus_queue.cc:237] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:14.946298 9564 raft_consensus.cc:2454] T 6aed24ef77e54c8b8ff09e6709a886e4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:14.948009 8890 catalog_manager.cc:5484] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d reported cstate change: term changed from 0 to 1, leader changed from <none> to bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65). New cstate: current_term: 1 leader_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } }
W20240502 14:01:15.064143 10591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:15.064433 10591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:15.067528 10591 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240502 14:01:15.151618 9698 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.151618 9433 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.151687 9564 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.151988 9168 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.152199 9830 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.152202 9301 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.152477 10621 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.152895 10589 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.153236 10589 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.153546 9168 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.153533 10589 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.153825 10589 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.154156 10397 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.154222 9036 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.155735 10427 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.162824 10397 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.167039 9565 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.167038 9696 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.167493 9829 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.168237 9302 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.168349 10361 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.168466 10399 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.168711 10397 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.170378 10396 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.213268 9302 raft_consensus.cc:3037] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.213403 9564 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.214082 9698 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.214462 9302 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.215090 10314 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.217536 10579 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.217847 10579 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.221541 9036 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.221975 10314 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.228509 9830 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.228925 10314 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.229036 9433 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.240209 10314 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 221.882 ms
time per row: 0.110941 ms
I20240502 14:01:15.332696 8890 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46030:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240502 14:01:15.333274 8890 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_3 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240502 14:01:15.339412 9148 tablet_service.cc:1467] Processing CreateTablet for tablet ada4c2cc1749411d8f41f882b7c64b07 (DEFAULT_TABLE table=rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.339754 9148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ada4c2cc1749411d8f41f882b7c64b07. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.339924 9018 tablet_service.cc:1467] Processing CreateTablet for tablet ada4c2cc1749411d8f41f882b7c64b07 (DEFAULT_TABLE table=rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.340171 9018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ada4c2cc1749411d8f41f882b7c64b07. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.342087 9546 tablet_service.cc:1467] Processing CreateTablet for tablet ada4c2cc1749411d8f41f882b7c64b07 (DEFAULT_TABLE table=rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.342197 10300 tablet_bootstrap.cc:492] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16: Bootstrap starting.
I20240502 14:01:15.342368 9546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ada4c2cc1749411d8f41f882b7c64b07. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.342715 10304 tablet_bootstrap.cc:492] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap starting.
I20240502 14:01:15.343070 10300 tablet_bootstrap.cc:654] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.343479 10304 tablet_bootstrap.cc:654] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.344241 9678 tablet_service.cc:1467] Processing CreateTablet for tablet ada4c2cc1749411d8f41f882b7c64b07 (DEFAULT_TABLE table=rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.344543 9678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ada4c2cc1749411d8f41f882b7c64b07. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.344556 10303 tablet_bootstrap.cc:492] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap starting.
I20240502 14:01:15.344614 9677 tablet_service.cc:1467] Processing CreateTablet for tablet 655c86a7f52a40c68c3e95cb12286c29 (DEFAULT_TABLE table=rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.344729 9545 tablet_service.cc:1467] Processing CreateTablet for tablet fed16a08a40a474a9159826ca8731acb (DEFAULT_TABLE table=rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.344858 9677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 655c86a7f52a40c68c3e95cb12286c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.344918 9545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fed16a08a40a474a9159826ca8731acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.345335 10303 tablet_bootstrap.cc:654] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.346747 9676 tablet_service.cc:1467] Processing CreateTablet for tablet fed16a08a40a474a9159826ca8731acb (DEFAULT_TABLE table=rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.346963 9676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fed16a08a40a474a9159826ca8731acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.346993 10303 tablet_bootstrap.cc:492] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca: No bootstrap required, opened a new log
I20240502 14:01:15.347069 10303 ts_tablet_manager.cc:1367] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:01:15.347038 9544 tablet_service.cc:1467] Processing CreateTablet for tablet 655c86a7f52a40c68c3e95cb12286c29 (DEFAULT_TABLE table=rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.347306 9544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 655c86a7f52a40c68c3e95cb12286c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.347715 10303 raft_consensus.cc:348] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.347828 10303 raft_consensus.cc:374] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.347858 10303 raft_consensus.cc:729] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.347985 10303 consensus_queue.cc:260] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.348307 10303 ts_tablet_manager.cc:1398] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:15.348477 10303 tablet_bootstrap.cc:492] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap starting.
I20240502 14:01:15.349037 10299 tablet_bootstrap.cc:492] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288: Bootstrap starting.
I20240502 14:01:15.349314 10303 tablet_bootstrap.cc:654] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.350024 10299 tablet_bootstrap.cc:654] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.351183 9280 tablet_service.cc:1467] Processing CreateTablet for tablet fed16a08a40a474a9159826ca8731acb (DEFAULT_TABLE table=rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.351207 9808 tablet_service.cc:1467] Processing CreateTablet for tablet ada4c2cc1749411d8f41f882b7c64b07 (DEFAULT_TABLE table=rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.351377 10304 tablet_bootstrap.cc:492] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d: No bootstrap required, opened a new log
I20240502 14:01:15.351382 9281 tablet_service.cc:1467] Processing CreateTablet for tablet 655c86a7f52a40c68c3e95cb12286c29 (DEFAULT_TABLE table=rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.351451 9280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fed16a08a40a474a9159826ca8731acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.351459 10304 ts_tablet_manager.cc:1367] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240502 14:01:15.351464 9808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ada4c2cc1749411d8f41f882b7c64b07. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.351581 9281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 655c86a7f52a40c68c3e95cb12286c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.351866 10304 raft_consensus.cc:348] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.351984 10304 raft_consensus.cc:374] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.352010 10304 raft_consensus.cc:729] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9e4f2f668d48ea87d5aad4a2fb569d, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.352118 10304 consensus_queue.cc:260] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.352454 10304 ts_tablet_manager.cc:1398] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:15.353368 9809 tablet_service.cc:1467] Processing CreateTablet for tablet 655c86a7f52a40c68c3e95cb12286c29 (DEFAULT_TABLE table=rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.353566 9809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 655c86a7f52a40c68c3e95cb12286c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.353649 10300 tablet_bootstrap.cc:492] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16: No bootstrap required, opened a new log
I20240502 14:01:15.353720 10305 tablet_bootstrap.cc:492] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b: Bootstrap starting.
I20240502 14:01:15.353750 10300 ts_tablet_manager.cc:1367] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.001s
I20240502 14:01:15.354194 10300 raft_consensus.cc:348] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.354321 10300 raft_consensus.cc:374] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.354349 10300 raft_consensus.cc:729] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83af1bc823a4f2ca531f0d6825f8e16, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.354477 10305 tablet_bootstrap.cc:654] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.354478 10300 consensus_queue.cc:260] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.354560 10301 tablet_bootstrap.cc:492] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f: Bootstrap starting.
I20240502 14:01:15.354765 10300 ts_tablet_manager.cc:1398] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:15.355289 10301 tablet_bootstrap.cc:654] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.355604 9810 tablet_service.cc:1467] Processing CreateTablet for tablet fed16a08a40a474a9159826ca8731acb (DEFAULT_TABLE table=rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.355705 10305 tablet_bootstrap.cc:492] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b: No bootstrap required, opened a new log
I20240502 14:01:15.355782 10305 ts_tablet_manager.cc:1367] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:15.355815 9810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fed16a08a40a474a9159826ca8731acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.356174 10301 tablet_bootstrap.cc:492] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f: No bootstrap required, opened a new log
I20240502 14:01:15.356153 10305 raft_consensus.cc:348] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.356242 10301 ts_tablet_manager.cc:1367] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:01:15.356274 10305 raft_consensus.cc:374] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.356307 10305 raft_consensus.cc:729] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.356438 10305 consensus_queue.cc:260] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.356693 10305 ts_tablet_manager.cc:1398] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:15.356698 10301 raft_consensus.cc:348] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.356809 10301 raft_consensus.cc:374] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.356835 10305 tablet_bootstrap.cc:492] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b: Bootstrap starting.
I20240502 14:01:15.356840 10301 raft_consensus.cc:729] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.356963 10301 consensus_queue.cc:260] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.357195 10301 ts_tablet_manager.cc:1398] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:15.357394 10301 tablet_bootstrap.cc:492] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f: Bootstrap starting.
I20240502 14:01:15.357652 10305 tablet_bootstrap.cc:654] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.351183 9282 tablet_service.cc:1467] Processing CreateTablet for tablet ada4c2cc1749411d8f41f882b7c64b07 (DEFAULT_TABLE table=rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.357896 9282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ada4c2cc1749411d8f41f882b7c64b07. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.358917 10299 tablet_bootstrap.cc:492] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288: No bootstrap required, opened a new log
I20240502 14:01:15.359012 10299 ts_tablet_manager.cc:1367] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 14:01:15.359309 10301 tablet_bootstrap.cc:654] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.359499 10299 raft_consensus.cc:348] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.359629 10299 raft_consensus.cc:374] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.359659 10299 raft_consensus.cc:729] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa836edab8d4853a7ca47a23117e288, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.359803 10299 consensus_queue.cc:260] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.359898 10305 tablet_bootstrap.cc:492] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b: No bootstrap required, opened a new log
I20240502 14:01:15.359974 10305 ts_tablet_manager.cc:1367] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:01:15.360139 10299 ts_tablet_manager.cc:1398] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:15.360299 10301 tablet_bootstrap.cc:492] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f: No bootstrap required, opened a new log
I20240502 14:01:15.360370 10299 tablet_bootstrap.cc:492] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288: Bootstrap starting.
I20240502 14:01:15.360420 10301 ts_tablet_manager.cc:1367] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 14:01:15.360474 10305 raft_consensus.cc:348] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.360600 10305 raft_consensus.cc:374] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.360630 10305 raft_consensus.cc:729] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.360741 10305 consensus_queue.cc:260] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.360844 10301 raft_consensus.cc:348] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.360949 10301 raft_consensus.cc:374] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.360975 10301 raft_consensus.cc:729] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.361029 10305 ts_tablet_manager.cc:1398] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:15.361074 10301 consensus_queue.cc:260] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.361172 10305 tablet_bootstrap.cc:492] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b: Bootstrap starting.
I20240502 14:01:15.361210 10299 tablet_bootstrap.cc:654] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.361392 10301 ts_tablet_manager.cc:1398] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:15.361573 10301 tablet_bootstrap.cc:492] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f: Bootstrap starting.
I20240502 14:01:15.362016 10305 tablet_bootstrap.cc:654] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.362414 10301 tablet_bootstrap.cc:654] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.363790 10299 tablet_bootstrap.cc:492] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288: No bootstrap required, opened a new log
I20240502 14:01:15.363865 10299 ts_tablet_manager.cc:1367] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:01:15.364466 10299 raft_consensus.cc:348] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.364559 10299 raft_consensus.cc:374] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.364579 10299 raft_consensus.cc:729] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa836edab8d4853a7ca47a23117e288, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.364647 9018 tablet_service.cc:1467] Processing CreateTablet for tablet fed16a08a40a474a9159826ca8731acb (DEFAULT_TABLE table=rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.364681 10299 consensus_queue.cc:260] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.364928 10299 ts_tablet_manager.cc:1398] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:15.364950 9018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fed16a08a40a474a9159826ca8731acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.365136 10299 tablet_bootstrap.cc:492] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288: Bootstrap starting.
I20240502 14:01:15.365401 9017 tablet_service.cc:1467] Processing CreateTablet for tablet 655c86a7f52a40c68c3e95cb12286c29 (DEFAULT_TABLE table=rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.365617 9017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 655c86a7f52a40c68c3e95cb12286c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.365940 10299 tablet_bootstrap.cc:654] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.367043 10304 tablet_bootstrap.cc:492] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap starting.
I20240502 14:01:15.367906 10304 tablet_bootstrap.cc:654] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.368764 9148 tablet_service.cc:1467] Processing CreateTablet for tablet fed16a08a40a474a9159826ca8731acb (DEFAULT_TABLE table=rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.368795 9150 tablet_service.cc:1467] Processing CreateTablet for tablet 655c86a7f52a40c68c3e95cb12286c29 (DEFAULT_TABLE table=rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.369004 9148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fed16a08a40a474a9159826ca8731acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.369215 9150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 655c86a7f52a40c68c3e95cb12286c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.369562 10304 tablet_bootstrap.cc:492] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d: No bootstrap required, opened a new log
I20240502 14:01:15.369634 10304 ts_tablet_manager.cc:1367] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:01:15.370000 10304 raft_consensus.cc:348] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.370111 10304 raft_consensus.cc:374] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.370141 10304 raft_consensus.cc:729] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9e4f2f668d48ea87d5aad4a2fb569d, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.370273 10304 consensus_queue.cc:260] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.370561 10304 ts_tablet_manager.cc:1398] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:15.370760 10304 tablet_bootstrap.cc:492] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap starting.
I20240502 14:01:15.371305 10300 tablet_bootstrap.cc:492] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16: Bootstrap starting.
I20240502 14:01:15.371515 10304 tablet_bootstrap.cc:654] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.372010 9413 tablet_service.cc:1467] Processing CreateTablet for tablet ada4c2cc1749411d8f41f882b7c64b07 (DEFAULT_TABLE table=rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.372010 9412 tablet_service.cc:1467] Processing CreateTablet for tablet fed16a08a40a474a9159826ca8731acb (DEFAULT_TABLE table=rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.372126 10300 tablet_bootstrap.cc:654] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.372319 9413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ada4c2cc1749411d8f41f882b7c64b07. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.374511 9414 tablet_service.cc:1467] Processing CreateTablet for tablet 655c86a7f52a40c68c3e95cb12286c29 (DEFAULT_TABLE table=rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.374861 9414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 655c86a7f52a40c68c3e95cb12286c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.374924 10306 tablet_bootstrap.cc:492] T ada4c2cc1749411d8f41f882b7c64b07 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:15.375383 9412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fed16a08a40a474a9159826ca8731acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.375725 10306 tablet_bootstrap.cc:654] T ada4c2cc1749411d8f41f882b7c64b07 P 1a490f170c614ee4900ef16c3287c7de: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.376641 10301 tablet_bootstrap.cc:492] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f: No bootstrap required, opened a new log
I20240502 14:01:15.376727 10301 ts_tablet_manager.cc:1367] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.001s
I20240502 14:01:15.377079 10305 tablet_bootstrap.cc:492] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b: No bootstrap required, opened a new log
I20240502 14:01:15.377164 10305 ts_tablet_manager.cc:1367] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 14:01:15.377172 10301 raft_consensus.cc:348] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.377369 10301 raft_consensus.cc:374] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.377413 10301 raft_consensus.cc:729] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.377555 10301 consensus_queue.cc:260] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.377632 10305 raft_consensus.cc:348] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.377754 10305 raft_consensus.cc:374] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.377795 10305 raft_consensus.cc:729] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.377897 10301 ts_tablet_manager.cc:1398] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:15.377919 10305 consensus_queue.cc:260] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.378067 10303 tablet_bootstrap.cc:492] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca: No bootstrap required, opened a new log
I20240502 14:01:15.378154 10303 ts_tablet_manager.cc:1367] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca: Time spent bootstrapping tablet: real 0.030s user 0.000s sys 0.002s
I20240502 14:01:15.378199 10305 ts_tablet_manager.cc:1398] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:15.378616 10303 raft_consensus.cc:348] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.378734 10303 raft_consensus.cc:374] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.378911 10303 raft_consensus.cc:729] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.379050 10303 consensus_queue.cc:260] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.379338 10303 ts_tablet_manager.cc:1398] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:15.379501 10303 tablet_bootstrap.cc:492] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap starting.
I20240502 14:01:15.380105 10306 tablet_bootstrap.cc:492] T ada4c2cc1749411d8f41f882b7c64b07 P 1a490f170c614ee4900ef16c3287c7de: No bootstrap required, opened a new log
I20240502 14:01:15.380177 10303 tablet_bootstrap.cc:654] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.380180 10306 ts_tablet_manager.cc:1367] T ada4c2cc1749411d8f41f882b7c64b07 P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 14:01:15.380767 10306 raft_consensus.cc:348] T ada4c2cc1749411d8f41f882b7c64b07 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.380856 10306 raft_consensus.cc:374] T ada4c2cc1749411d8f41f882b7c64b07 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.380889 10306 raft_consensus.cc:729] T ada4c2cc1749411d8f41f882b7c64b07 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.381003 10306 consensus_queue.cc:260] T ada4c2cc1749411d8f41f882b7c64b07 P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.381148 10303 tablet_bootstrap.cc:492] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca: No bootstrap required, opened a new log
I20240502 14:01:15.381232 10303 ts_tablet_manager.cc:1367] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:01:15.381253 10306 ts_tablet_manager.cc:1398] T ada4c2cc1749411d8f41f882b7c64b07 P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:15.381408 10306 tablet_bootstrap.cc:492] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:15.382242 10306 tablet_bootstrap.cc:654] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.382230 10303 raft_consensus.cc:348] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.382329 10303 raft_consensus.cc:374] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.382360 10303 raft_consensus.cc:729] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.382488 10303 consensus_queue.cc:260] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.382804 10303 ts_tablet_manager.cc:1398] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:01:15.383275 10306 tablet_bootstrap.cc:492] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de: No bootstrap required, opened a new log
I20240502 14:01:15.383352 10306 ts_tablet_manager.cc:1367] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:15.383733 10306 raft_consensus.cc:348] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.383842 10306 raft_consensus.cc:374] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.383888 10306 raft_consensus.cc:729] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.384027 10306 consensus_queue.cc:260] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.384656 10299 tablet_bootstrap.cc:492] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288: No bootstrap required, opened a new log
I20240502 14:01:15.384755 10299 ts_tablet_manager.cc:1367] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.000s
I20240502 14:01:15.385208 10299 raft_consensus.cc:348] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.385355 10299 raft_consensus.cc:374] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.385392 10299 raft_consensus.cc:729] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa836edab8d4853a7ca47a23117e288, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.385541 10299 consensus_queue.cc:260] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.385720 10306 ts_tablet_manager.cc:1398] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:15.385877 10306 tablet_bootstrap.cc:492] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:15.385939 10299 ts_tablet_manager.cc:1398] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:15.386596 10306 tablet_bootstrap.cc:654] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.392652 10300 tablet_bootstrap.cc:492] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16: No bootstrap required, opened a new log
I20240502 14:01:15.392750 10300 ts_tablet_manager.cc:1367] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20240502 14:01:15.393146 10300 raft_consensus.cc:348] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.393246 10300 raft_consensus.cc:374] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.393278 10300 raft_consensus.cc:729] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83af1bc823a4f2ca531f0d6825f8e16, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.393409 10300 consensus_queue.cc:260] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.393654 10300 ts_tablet_manager.cc:1398] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:15.393793 10300 tablet_bootstrap.cc:492] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16: Bootstrap starting.
I20240502 14:01:15.394708 10300 tablet_bootstrap.cc:654] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.395869 10300 tablet_bootstrap.cc:492] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16: No bootstrap required, opened a new log
I20240502 14:01:15.395944 10300 ts_tablet_manager.cc:1367] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:15.396265 10300 raft_consensus.cc:348] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.396435 10300 raft_consensus.cc:374] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.396469 10300 raft_consensus.cc:729] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83af1bc823a4f2ca531f0d6825f8e16, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.396605 10300 consensus_queue.cc:260] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.396764 10304 tablet_bootstrap.cc:492] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d: No bootstrap required, opened a new log
I20240502 14:01:15.396862 10304 ts_tablet_manager.cc:1367] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20240502 14:01:15.396934 10300 ts_tablet_manager.cc:1398] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:15.397362 10304 raft_consensus.cc:348] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.397472 10304 raft_consensus.cc:374] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.397495 10304 raft_consensus.cc:729] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9e4f2f668d48ea87d5aad4a2fb569d, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.397624 10304 consensus_queue.cc:260] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.397892 10304 ts_tablet_manager.cc:1398] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:15.400636 10306 tablet_bootstrap.cc:492] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de: No bootstrap required, opened a new log
I20240502 14:01:15.400768 10306 ts_tablet_manager.cc:1367] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 14:01:15.401177 10306 raft_consensus.cc:348] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.401275 10306 raft_consensus.cc:374] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.401295 10306 raft_consensus.cc:729] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.401378 10306 consensus_queue.cc:260] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.401614 10306 ts_tablet_manager.cc:1398] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:15.406203 10439 raft_consensus.cc:482] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:15.406307 10439 raft_consensus.cc:504] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.406922 10439 leader_election.cc:288] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375)
I20240502 14:01:15.407171 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "655c86a7f52a40c68c3e95cb12286c29" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" is_pre_election: true
I20240502 14:01:15.407330 9036 raft_consensus.cc:2454] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:15.407390 9698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "655c86a7f52a40c68c3e95cb12286c29" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288" is_pre_election: true
I20240502 14:01:15.407500 9698 raft_consensus.cc:2454] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:15.407549 9168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "655c86a7f52a40c68c3e95cb12286c29" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" is_pre_election: true
I20240502 14:01:15.407665 9168 raft_consensus.cc:2454] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:15.407924 9765 leader_election.cc:302] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0488b0e4ed994db19d40aed8c15f945b, bc9e4f2f668d48ea87d5aad4a2fb569d, d83af1bc823a4f2ca531f0d6825f8e16, efa836edab8d4853a7ca47a23117e288; no voters:
I20240502 14:01:15.408008 9301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "655c86a7f52a40c68c3e95cb12286c29" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" is_pre_election: true
I20240502 14:01:15.408085 10439 raft_consensus.cc:2781] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:15.408130 10439 raft_consensus.cc:482] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:15.408124 9566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "655c86a7f52a40c68c3e95cb12286c29" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca" is_pre_election: true
I20240502 14:01:15.408159 9301 raft_consensus.cc:2454] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:15.408175 10439 raft_consensus.cc:3037] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.408263 9566 raft_consensus.cc:2454] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:15.408536 9434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "655c86a7f52a40c68c3e95cb12286c29" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de" is_pre_election: true
I20240502 14:01:15.408684 9434 raft_consensus.cc:2454] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:15.409248 10439 raft_consensus.cc:504] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.409878 10439 leader_election.cc:288] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 election: Requested vote from peers d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375)
I20240502 14:01:15.410055 9301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "655c86a7f52a40c68c3e95cb12286c29" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f"
I20240502 14:01:15.410133 9698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "655c86a7f52a40c68c3e95cb12286c29" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288"
I20240502 14:01:15.410184 9301 raft_consensus.cc:3037] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.410260 9698 raft_consensus.cc:3037] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.410424 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "655c86a7f52a40c68c3e95cb12286c29" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d"
I20240502 14:01:15.410553 9036 raft_consensus.cc:3037] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.411047 9301 raft_consensus.cc:2454] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:15.411291 9698 raft_consensus.cc:2454] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:15.411394 9168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "655c86a7f52a40c68c3e95cb12286c29" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16"
I20240502 14:01:15.411520 9036 raft_consensus.cc:2454] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:15.411538 9168 raft_consensus.cc:3037] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.411918 9766 leader_election.cc:302] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0488b0e4ed994db19d40aed8c15f945b, bc9e4f2f668d48ea87d5aad4a2fb569d, ccfb5bbc1bb54793812d11170ac2d25f, efa836edab8d4853a7ca47a23117e288; no voters:
I20240502 14:01:15.411986 9566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "655c86a7f52a40c68c3e95cb12286c29" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca"
I20240502 14:01:15.412094 9566 raft_consensus.cc:3037] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.412104 10439 raft_consensus.cc:2781] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:15.412178 10439 raft_consensus.cc:686] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Becoming Leader. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Running, Role: LEADER
I20240502 14:01:15.412346 10439 consensus_queue.cc:237] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.412533 9168 raft_consensus.cc:2454] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:15.413128 9566 raft_consensus.cc:2454] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:15.413503 9434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "655c86a7f52a40c68c3e95cb12286c29" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de"
I20240502 14:01:15.413611 9434 raft_consensus.cc:3037] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.414453 9434 raft_consensus.cc:2454] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:15.414527 8890 catalog_manager.cc:5484] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71). New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:15.415462 10548 raft_consensus.cc:482] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:15.415555 10548 raft_consensus.cc:504] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.416071 10548 leader_election.cc:288] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:15.416383 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fed16a08a40a474a9159826ca8731acb" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" is_pre_election: true
I20240502 14:01:15.416491 9036 raft_consensus.cc:2454] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 0.
I20240502 14:01:15.416543 9828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fed16a08a40a474a9159826ca8731acb" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b" is_pre_election: true
I20240502 14:01:15.416579 9566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fed16a08a40a474a9159826ca8731acb" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca" is_pre_election: true
I20240502 14:01:15.416664 9828 raft_consensus.cc:2454] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 0.
I20240502 14:01:15.416687 9566 raft_consensus.cc:2454] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 0.
I20240502 14:01:15.416675 9168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fed16a08a40a474a9159826ca8731acb" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" is_pre_election: true
I20240502 14:01:15.416764 9168 raft_consensus.cc:2454] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 0.
I20240502 14:01:15.416978 9434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fed16a08a40a474a9159826ca8731acb" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de" is_pre_election: true
I20240502 14:01:15.417088 9434 raft_consensus.cc:2454] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 0.
I20240502 14:01:15.417414 9237 leader_election.cc:302] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0488b0e4ed994db19d40aed8c15f945b, bc9e4f2f668d48ea87d5aad4a2fb569d, ccfb5bbc1bb54793812d11170ac2d25f, d83af1bc823a4f2ca531f0d6825f8e16; no voters:
I20240502 14:01:15.417611 10548 raft_consensus.cc:2781] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:15.417621 9698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fed16a08a40a474a9159826ca8731acb" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288" is_pre_election: true
I20240502 14:01:15.417676 10548 raft_consensus.cc:482] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:15.417729 10548 raft_consensus.cc:3037] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.417749 9698 raft_consensus.cc:2454] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 0.
I20240502 14:01:15.418617 10548 raft_consensus.cc:504] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.419085 10548 leader_election.cc:288] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [CANDIDATE]: Term 1 election: Requested vote from peers bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:15.419225 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fed16a08a40a474a9159826ca8731acb" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d"
I20240502 14:01:15.419278 9566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fed16a08a40a474a9159826ca8731acb" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca"
I20240502 14:01:15.419342 9036 raft_consensus.cc:3037] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.419384 9566 raft_consensus.cc:3037] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.419549 9434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fed16a08a40a474a9159826ca8731acb" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de"
I20240502 14:01:15.419546 9828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fed16a08a40a474a9159826ca8731acb" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b"
I20240502 14:01:15.419633 9434 raft_consensus.cc:3037] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.419641 9828 raft_consensus.cc:3037] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.420233 9036 raft_consensus.cc:2454] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 1.
I20240502 14:01:15.420243 9566 raft_consensus.cc:2454] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 1.
I20240502 14:01:15.420473 9434 raft_consensus.cc:2454] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 1.
I20240502 14:01:15.420650 9828 raft_consensus.cc:2454] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 1.
I20240502 14:01:15.420668 9168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fed16a08a40a474a9159826ca8731acb" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16"
I20240502 14:01:15.420753 9168 raft_consensus.cc:3037] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.420728 9237 leader_election.cc:302] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1a490f170c614ee4900ef16c3287c7de, bc9e4f2f668d48ea87d5aad4a2fb569d, ccfb5bbc1bb54793812d11170ac2d25f, d0f45862d7f14ce1976c1a37842f25ca; no voters:
I20240502 14:01:15.420913 10548 raft_consensus.cc:2781] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:15.420990 10548 raft_consensus.cc:686] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Becoming Leader. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Running, Role: LEADER
I20240502 14:01:15.421142 10548 consensus_queue.cc:237] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.421468 9168 raft_consensus.cc:2454] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 1.
I20240502 14:01:15.421661 9698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fed16a08a40a474a9159826ca8731acb" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288"
I20240502 14:01:15.421773 9698 raft_consensus.cc:3037] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.422641 9698 raft_consensus.cc:2454] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 1.
I20240502 14:01:15.423166 8890 catalog_manager.cc:5484] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: term changed from 0 to 1, leader changed from <none> to ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67). New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:15.490650 10638 raft_consensus.cc:482] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:15.490803 10638 raft_consensus.cc:504] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.491437 10638 leader_election.cc:288] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749), 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:15.491809 9828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ada4c2cc1749411d8f41f882b7c64b07" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b" is_pre_election: true
I20240502 14:01:15.491812 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ada4c2cc1749411d8f41f882b7c64b07" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" is_pre_election: true
I20240502 14:01:15.491950 9828 raft_consensus.cc:2454] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 0.
I20240502 14:01:15.491973 9036 raft_consensus.cc:2454] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 0.
I20240502 14:01:15.491977 9434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ada4c2cc1749411d8f41f882b7c64b07" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de" is_pre_election: true
I20240502 14:01:15.492095 9434 raft_consensus.cc:2454] T ada4c2cc1749411d8f41f882b7c64b07 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 0.
I20240502 14:01:15.492466 9105 leader_election.cc:302] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0488b0e4ed994db19d40aed8c15f945b, 1a490f170c614ee4900ef16c3287c7de, bc9e4f2f668d48ea87d5aad4a2fb569d, d83af1bc823a4f2ca531f0d6825f8e16; no voters:
I20240502 14:01:15.492471 9698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ada4c2cc1749411d8f41f882b7c64b07" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288" is_pre_election: true
I20240502 14:01:15.492645 10638 raft_consensus.cc:2781] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:15.492631 9301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ada4c2cc1749411d8f41f882b7c64b07" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" is_pre_election: true
I20240502 14:01:15.492702 10638 raft_consensus.cc:482] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:15.492750 10638 raft_consensus.cc:3037] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.492684 9698 raft_consensus.cc:2454] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 0.
I20240502 14:01:15.492767 9301 raft_consensus.cc:2454] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 0.
I20240502 14:01:15.492735 9566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ada4c2cc1749411d8f41f882b7c64b07" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca" is_pre_election: true
I20240502 14:01:15.493054 9566 raft_consensus.cc:2454] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 0.
I20240502 14:01:15.493788 10638 raft_consensus.cc:504] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.494349 10638 leader_election.cc:288] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [CANDIDATE]: Term 1 election: Requested vote from peers bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749), 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:15.494468 9301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ada4c2cc1749411d8f41f882b7c64b07" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f"
I20240502 14:01:15.494534 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ada4c2cc1749411d8f41f882b7c64b07" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d"
I20240502 14:01:15.494567 9301 raft_consensus.cc:3037] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.494597 9434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ada4c2cc1749411d8f41f882b7c64b07" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de"
I20240502 14:01:15.494637 9036 raft_consensus.cc:3037] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.494710 9434 raft_consensus.cc:3037] T ada4c2cc1749411d8f41f882b7c64b07 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.494884 9566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ada4c2cc1749411d8f41f882b7c64b07" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca"
I20240502 14:01:15.495023 9566 raft_consensus.cc:3037] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.495523 9301 raft_consensus.cc:2454] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 1.
I20240502 14:01:15.495523 9036 raft_consensus.cc:2454] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 1.
I20240502 14:01:15.495764 9434 raft_consensus.cc:2454] T ada4c2cc1749411d8f41f882b7c64b07 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 1.
I20240502 14:01:15.496065 9698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ada4c2cc1749411d8f41f882b7c64b07" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288"
I20240502 14:01:15.496109 9828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ada4c2cc1749411d8f41f882b7c64b07" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b"
I20240502 14:01:15.496196 9698 raft_consensus.cc:3037] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.496225 9828 raft_consensus.cc:3037] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.496266 9105 leader_election.cc:302] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1a490f170c614ee4900ef16c3287c7de, bc9e4f2f668d48ea87d5aad4a2fb569d, ccfb5bbc1bb54793812d11170ac2d25f, d83af1bc823a4f2ca531f0d6825f8e16; no voters:
I20240502 14:01:15.496446 10638 raft_consensus.cc:2781] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:15.496522 10638 raft_consensus.cc:686] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Becoming Leader. State: Replica: d83af1bc823a4f2ca531f0d6825f8e16, State: Running, Role: LEADER
I20240502 14:01:15.496543 9566 raft_consensus.cc:2454] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 1.
I20240502 14:01:15.496662 10638 consensus_queue.cc:237] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.497314 9828 raft_consensus.cc:2454] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 1.
I20240502 14:01:15.497356 9698 raft_consensus.cc:2454] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 1.
I20240502 14:01:15.498975 8890 catalog_manager.cc:5484] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: term changed from 0 to 1, leader changed from <none> to d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66). New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } }
W20240502 14:01:15.626917 10673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:15.627161 10673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:15.630472 10673 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20240502 14:01:15.709494 9434 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.709949 9698 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.710790 9566 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.711377 9036 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.711728 9168 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.711982 10546 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.712919 9828 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.712982 9301 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.713358 10439 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.713654 10439 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.713698 9698 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.713840 9829 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.713939 10637 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.716598 9433 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.717020 9565 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.717083 9434 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.717449 10546 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.717772 10546 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.718015 10546 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.724645 9036 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.724671 10636 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.724937 10636 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.725088 10638 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.725520 9168 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:15.725853 10523 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.725991 9564 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:15.726321 9302 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:15.726552 10637 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.727100 10548 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.727373 10548 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.727447 9698 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:15.729314 10638 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.730130 10715 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.739303 10523 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.740172 9036 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:15.740620 10523 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:15.796069 10418 mvcc.cc:205] Tried to move back new op lower bound from 7023241116505796608 to 7023241115327983616. Current Snapshot: MvccSnapshot[applied={T|T < 7023241116505796608}]
I20240502 14:01:15.799861 10421 mvcc.cc:205] Tried to move back new op lower bound from 7023241116540334080 to 7023241115292364800. Current Snapshot: MvccSnapshot[applied={T|T < 7023241116500725760}]
I20240502 14:01:15.804617 10432 mvcc.cc:205] Tried to move back new op lower bound from 7023241116505796608 to 7023241115327983616. Current Snapshot: MvccSnapshot[applied={T|T < 7023241116505796608}]
I20240502 14:01:15.816452 10412 mvcc.cc:205] Tried to move back new op lower bound from 7023241116584845312 to 7023241115327983616. Current Snapshot: MvccSnapshot[applied={T|T < 7023241116505796608}]
I20240502 14:01:15.822968 10392 mvcc.cc:205] Tried to move back new op lower bound from 7023241116540334080 to 7023241115292364800. Current Snapshot: MvccSnapshot[applied={T|T < 7023241116500725760}]
I20240502 14:01:15.838084 10749 mvcc.cc:205] Tried to move back new op lower bound from 7023241116505796608 to 7023241115327983616. Current Snapshot: MvccSnapshot[applied={T|T < 7023241116505796608}]
INSERT report
rows total: 2000
time total: 210.406 ms
time per row: 0.105203 ms
I20240502 14:01:15.890765 8894 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46030:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240502 14:01:15.891274 8894 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_4 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240502 14:01:15.896497 9017 tablet_service.cc:1467] Processing CreateTablet for tablet 21d4acfb937f4dd4b992c338fc6b3237 (DEFAULT_TABLE table=rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.896682 9148 tablet_service.cc:1467] Processing CreateTablet for tablet 21d4acfb937f4dd4b992c338fc6b3237 (DEFAULT_TABLE table=rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.896818 9017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d4acfb937f4dd4b992c338fc6b3237. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.896865 9282 tablet_service.cc:1467] Processing CreateTablet for tablet 21d4acfb937f4dd4b992c338fc6b3237 (DEFAULT_TABLE table=rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.896957 9148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d4acfb937f4dd4b992c338fc6b3237. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.897140 9282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d4acfb937f4dd4b992c338fc6b3237. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.899145 10304 tablet_bootstrap.cc:492] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap starting.
I20240502 14:01:15.899283 10300 tablet_bootstrap.cc:492] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16: Bootstrap starting.
I20240502 14:01:15.900015 10304 tablet_bootstrap.cc:654] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.900024 10300 tablet_bootstrap.cc:654] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.901448 9412 tablet_service.cc:1467] Processing CreateTablet for tablet 21d4acfb937f4dd4b992c338fc6b3237 (DEFAULT_TABLE table=rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.901598 10300 tablet_bootstrap.cc:492] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16: No bootstrap required, opened a new log
I20240502 14:01:15.901679 10300 ts_tablet_manager.cc:1367] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:15.901703 9412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d4acfb937f4dd4b992c338fc6b3237. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.902060 10300 raft_consensus.cc:348] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.902176 10300 raft_consensus.cc:374] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.902206 10300 raft_consensus.cc:729] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83af1bc823a4f2ca531f0d6825f8e16, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.902314 10300 consensus_queue.cc:260] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.902606 10300 ts_tablet_manager.cc:1398] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:15.902938 9544 tablet_service.cc:1467] Processing CreateTablet for tablet 21d4acfb937f4dd4b992c338fc6b3237 (DEFAULT_TABLE table=rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.903187 9544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d4acfb937f4dd4b992c338fc6b3237. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.904006 10756 tablet_bootstrap.cc:492] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f: Bootstrap starting.
I20240502 14:01:15.904860 10756 tablet_bootstrap.cc:654] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.905189 10760 tablet_bootstrap.cc:492] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap starting.
I20240502 14:01:15.905927 10304 tablet_bootstrap.cc:492] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d: No bootstrap required, opened a new log
I20240502 14:01:15.906014 10304 ts_tablet_manager.cc:1367] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 14:01:15.906101 10760 tablet_bootstrap.cc:654] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.906407 10304 raft_consensus.cc:348] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.906539 10304 raft_consensus.cc:374] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.906584 10304 raft_consensus.cc:729] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9e4f2f668d48ea87d5aad4a2fb569d, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.906733 10304 consensus_queue.cc:260] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.907042 10304 ts_tablet_manager.cc:1398] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:15.907196 10760 tablet_bootstrap.cc:492] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca: No bootstrap required, opened a new log
I20240502 14:01:15.907272 10760 ts_tablet_manager.cc:1367] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:15.907666 10760 raft_consensus.cc:348] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.907709 9676 tablet_service.cc:1467] Processing CreateTablet for tablet 21d4acfb937f4dd4b992c338fc6b3237 (DEFAULT_TABLE table=rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.907775 10760 raft_consensus.cc:374] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.907809 10760 raft_consensus.cc:729] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.907997 9676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d4acfb937f4dd4b992c338fc6b3237. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.907949 10760 consensus_queue.cc:260] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.908383 9282 tablet_service.cc:1467] Processing CreateTablet for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a (DEFAULT_TABLE table=rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.908603 9282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.908900 10760 ts_tablet_manager.cc:1398] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:15.909571 10759 tablet_bootstrap.cc:492] T 21d4acfb937f4dd4b992c338fc6b3237 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:15.910272 10763 tablet_bootstrap.cc:492] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288: Bootstrap starting.
I20240502 14:01:15.910445 10759 tablet_bootstrap.cc:654] T 21d4acfb937f4dd4b992c338fc6b3237 P 1a490f170c614ee4900ef16c3287c7de: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.911507 10756 tablet_bootstrap.cc:492] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f: No bootstrap required, opened a new log
I20240502 14:01:15.911459 9412 tablet_service.cc:1467] Processing CreateTablet for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a (DEFAULT_TABLE table=rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.911590 10756 ts_tablet_manager.cc:1367] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:01:15.911693 9412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.911834 9414 tablet_service.cc:1467] Processing CreateTablet for tablet 29eec0daa5d34e40a4ad82607b7ded90 (DEFAULT_TABLE table=rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.911995 10756 raft_consensus.cc:348] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.912108 10756 raft_consensus.cc:374] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.912137 10756 raft_consensus.cc:729] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.912248 10756 consensus_queue.cc:260] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.912431 9414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29eec0daa5d34e40a4ad82607b7ded90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.912549 10756 ts_tablet_manager.cc:1398] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:15.912573 10763 tablet_bootstrap.cc:654] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.912721 10756 tablet_bootstrap.cc:492] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f: Bootstrap starting.
I20240502 14:01:15.913486 10756 tablet_bootstrap.cc:654] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.913489 10763 tablet_bootstrap.cc:492] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288: No bootstrap required, opened a new log
I20240502 14:01:15.913573 10763 ts_tablet_manager.cc:1367] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:01:15.913980 10763 raft_consensus.cc:348] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.914098 10763 raft_consensus.cc:374] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.914137 10763 raft_consensus.cc:729] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa836edab8d4853a7ca47a23117e288, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.914247 10763 consensus_queue.cc:260] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.914426 9018 tablet_service.cc:1467] Processing CreateTablet for tablet 29eec0daa5d34e40a4ad82607b7ded90 (DEFAULT_TABLE table=rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.914547 10763 ts_tablet_manager.cc:1398] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:15.914654 9018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29eec0daa5d34e40a4ad82607b7ded90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.915259 9809 tablet_service.cc:1467] Processing CreateTablet for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a (DEFAULT_TABLE table=rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.915527 9809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.916309 9282 tablet_service.cc:1467] Processing CreateTablet for tablet 29eec0daa5d34e40a4ad82607b7ded90 (DEFAULT_TABLE table=rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.916545 9282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29eec0daa5d34e40a4ad82607b7ded90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.916822 9017 tablet_service.cc:1467] Processing CreateTablet for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a (DEFAULT_TABLE table=rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.917025 9017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.918013 9808 tablet_service.cc:1467] Processing CreateTablet for tablet 29eec0daa5d34e40a4ad82607b7ded90 (DEFAULT_TABLE table=rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.918231 9808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29eec0daa5d34e40a4ad82607b7ded90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.918975 10767 tablet_bootstrap.cc:492] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b: Bootstrap starting.
I20240502 14:01:15.919150 10756 tablet_bootstrap.cc:492] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f: No bootstrap required, opened a new log
I20240502 14:01:15.919098 9150 tablet_service.cc:1467] Processing CreateTablet for tablet 29eec0daa5d34e40a4ad82607b7ded90 (DEFAULT_TABLE table=rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.919216 10756 ts_tablet_manager.cc:1367] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 14:01:15.919329 9150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29eec0daa5d34e40a4ad82607b7ded90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.919595 10756 raft_consensus.cc:348] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.919729 10756 raft_consensus.cc:374] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.919772 10767 tablet_bootstrap.cc:654] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.919768 10756 raft_consensus.cc:729] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.919894 10756 consensus_queue.cc:260] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.920162 10756 ts_tablet_manager.cc:1398] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:15.920321 10756 tablet_bootstrap.cc:492] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f: Bootstrap starting.
I20240502 14:01:15.920799 10759 tablet_bootstrap.cc:492] T 21d4acfb937f4dd4b992c338fc6b3237 P 1a490f170c614ee4900ef16c3287c7de: No bootstrap required, opened a new log
I20240502 14:01:15.920890 10759 ts_tablet_manager.cc:1367] T 21d4acfb937f4dd4b992c338fc6b3237 P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240502 14:01:15.921166 10756 tablet_bootstrap.cc:654] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.921337 10300 tablet_bootstrap.cc:492] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16: Bootstrap starting.
I20240502 14:01:15.921536 9148 tablet_service.cc:1467] Processing CreateTablet for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a (DEFAULT_TABLE table=rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.921753 9148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.922114 10300 tablet_bootstrap.cc:654] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.922138 10756 tablet_bootstrap.cc:492] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f: No bootstrap required, opened a new log
I20240502 14:01:15.922204 10756 ts_tablet_manager.cc:1367] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:15.922605 10756 raft_consensus.cc:348] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.922732 10756 raft_consensus.cc:374] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.922776 10756 raft_consensus.cc:729] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.922706 10759 raft_consensus.cc:348] T 21d4acfb937f4dd4b992c338fc6b3237 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.922829 10759 raft_consensus.cc:374] T 21d4acfb937f4dd4b992c338fc6b3237 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.922868 10759 raft_consensus.cc:729] T 21d4acfb937f4dd4b992c338fc6b3237 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.922921 10756 consensus_queue.cc:260] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.922998 10759 consensus_queue.cc:260] T 21d4acfb937f4dd4b992c338fc6b3237 P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.923223 10300 tablet_bootstrap.cc:492] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16: No bootstrap required, opened a new log
I20240502 14:01:15.923262 10756 ts_tablet_manager.cc:1398] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:15.923293 10300 ts_tablet_manager.cc:1367] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:15.923357 10759 ts_tablet_manager.cc:1398] T 21d4acfb937f4dd4b992c338fc6b3237 P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:15.923511 10759 tablet_bootstrap.cc:492] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:15.923689 10300 raft_consensus.cc:348] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.923806 10300 raft_consensus.cc:374] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.923780 9810 tablet_service.cc:1467] Processing CreateTablet for tablet 21d4acfb937f4dd4b992c338fc6b3237 (DEFAULT_TABLE table=rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.923837 10300 raft_consensus.cc:729] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83af1bc823a4f2ca531f0d6825f8e16, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.923956 9810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d4acfb937f4dd4b992c338fc6b3237. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.923962 10300 consensus_queue.cc:260] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.924216 10300 ts_tablet_manager.cc:1398] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:15.924329 10759 tablet_bootstrap.cc:654] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.924412 10300 tablet_bootstrap.cc:492] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16: Bootstrap starting.
I20240502 14:01:15.924492 10304 tablet_bootstrap.cc:492] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap starting.
I20240502 14:01:15.925262 10300 tablet_bootstrap.cc:654] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.925379 10304 tablet_bootstrap.cc:654] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.925457 10767 tablet_bootstrap.cc:492] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b: No bootstrap required, opened a new log
I20240502 14:01:15.925539 10767 ts_tablet_manager.cc:1367] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 14:01:15.925942 10767 raft_consensus.cc:348] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.926062 10767 raft_consensus.cc:374] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.926092 10767 raft_consensus.cc:729] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.926344 10767 consensus_queue.cc:260] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.926985 9676 tablet_service.cc:1467] Processing CreateTablet for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a (DEFAULT_TABLE table=rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.927220 9676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.928030 10767 ts_tablet_manager.cc:1398] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:15.928085 10300 tablet_bootstrap.cc:492] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16: No bootstrap required, opened a new log
I20240502 14:01:15.928150 10300 ts_tablet_manager.cc:1367] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:01:15.928186 10767 tablet_bootstrap.cc:492] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b: Bootstrap starting.
I20240502 14:01:15.928543 10300 raft_consensus.cc:348] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.928648 10300 raft_consensus.cc:374] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.928678 10300 raft_consensus.cc:729] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83af1bc823a4f2ca531f0d6825f8e16, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.928786 10300 consensus_queue.cc:260] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.928934 10767 tablet_bootstrap.cc:654] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.929019 10763 tablet_bootstrap.cc:492] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288: Bootstrap starting.
I20240502 14:01:15.929112 10300 ts_tablet_manager.cc:1398] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:15.929167 9678 tablet_service.cc:1467] Processing CreateTablet for tablet 29eec0daa5d34e40a4ad82607b7ded90 (DEFAULT_TABLE table=rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.929376 9678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29eec0daa5d34e40a4ad82607b7ded90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.929817 10763 tablet_bootstrap.cc:654] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.931807 9544 tablet_service.cc:1467] Processing CreateTablet for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a (DEFAULT_TABLE table=rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.932070 9544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.932989 10304 tablet_bootstrap.cc:492] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d: No bootstrap required, opened a new log
I20240502 14:01:15.933071 10304 ts_tablet_manager.cc:1367] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 14:01:15.933490 10304 raft_consensus.cc:348] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.933568 10763 tablet_bootstrap.cc:492] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288: No bootstrap required, opened a new log
I20240502 14:01:15.933604 10304 raft_consensus.cc:374] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.933635 10304 raft_consensus.cc:729] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9e4f2f668d48ea87d5aad4a2fb569d, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.933650 10763 ts_tablet_manager.cc:1367] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:01:15.933656 10759 tablet_bootstrap.cc:492] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de: No bootstrap required, opened a new log
I20240502 14:01:15.933722 10759 ts_tablet_manager.cc:1367] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20240502 14:01:15.933754 10304 consensus_queue.cc:260] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.934065 10304 ts_tablet_manager.cc:1398] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:15.934059 10763 raft_consensus.cc:348] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.934098 10759 raft_consensus.cc:348] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.934167 10763 raft_consensus.cc:374] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.934199 10759 raft_consensus.cc:374] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.934201 10763 raft_consensus.cc:729] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa836edab8d4853a7ca47a23117e288, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.934212 10304 tablet_bootstrap.cc:492] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap starting.
I20240502 14:01:15.934231 10759 raft_consensus.cc:729] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.934247 9545 tablet_service.cc:1467] Processing CreateTablet for tablet 29eec0daa5d34e40a4ad82607b7ded90 (DEFAULT_TABLE table=rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:15.934343 10763 consensus_queue.cc:260] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.934365 10759 consensus_queue.cc:260] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.934480 9545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29eec0daa5d34e40a4ad82607b7ded90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:15.934609 10759 ts_tablet_manager.cc:1398] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:15.934621 10763 ts_tablet_manager.cc:1398] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:15.934762 10759 tablet_bootstrap.cc:492] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:15.934762 10763 tablet_bootstrap.cc:492] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288: Bootstrap starting.
I20240502 14:01:15.935542 10763 tablet_bootstrap.cc:654] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.935542 10759 tablet_bootstrap.cc:654] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.936385 10760 tablet_bootstrap.cc:492] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap starting.
I20240502 14:01:15.936400 10767 tablet_bootstrap.cc:492] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b: No bootstrap required, opened a new log
I20240502 14:01:15.936476 10767 ts_tablet_manager.cc:1367] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:01:15.936820 10767 raft_consensus.cc:348] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.936900 10767 raft_consensus.cc:374] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.936920 10767 raft_consensus.cc:729] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.937021 10767 consensus_queue.cc:260] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.937207 10760 tablet_bootstrap.cc:654] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d0f45862d7f14ce1976c1a37842f25ca: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.937309 10767 ts_tablet_manager.cc:1398] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:15.937472 10767 tablet_bootstrap.cc:492] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b: Bootstrap starting.
I20240502 14:01:15.938251 10767 tablet_bootstrap.cc:654] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.939210 10767 tablet_bootstrap.cc:492] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b: No bootstrap required, opened a new log
I20240502 14:01:15.939304 10767 ts_tablet_manager.cc:1367] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:15.939711 10767 raft_consensus.cc:348] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.939836 10304 tablet_bootstrap.cc:654] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.939849 10767 raft_consensus.cc:374] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.939880 10767 raft_consensus.cc:729] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.940042 10767 consensus_queue.cc:260] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.940433 10767 ts_tablet_manager.cc:1398] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:15.941115 10304 tablet_bootstrap.cc:492] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d: No bootstrap required, opened a new log
I20240502 14:01:15.941161 10759 tablet_bootstrap.cc:492] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de: No bootstrap required, opened a new log
I20240502 14:01:15.941196 10304 ts_tablet_manager.cc:1367] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20240502 14:01:15.941238 10759 ts_tablet_manager.cc:1367] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20240502 14:01:15.941541 10763 tablet_bootstrap.cc:492] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288: No bootstrap required, opened a new log
I20240502 14:01:15.941627 10763 ts_tablet_manager.cc:1367] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 14:01:15.941640 10304 raft_consensus.cc:348] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.941658 10759 raft_consensus.cc:348] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.941773 10759 raft_consensus.cc:374] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.941766 10304 raft_consensus.cc:374] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.941813 10759 raft_consensus.cc:729] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.941813 10304 raft_consensus.cc:729] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9e4f2f668d48ea87d5aad4a2fb569d, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.941923 10759 consensus_queue.cc:260] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.941952 10304 consensus_queue.cc:260] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.942034 10763 raft_consensus.cc:348] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.942138 10763 raft_consensus.cc:374] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.942170 10763 raft_consensus.cc:729] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa836edab8d4853a7ca47a23117e288, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.942217 10759 ts_tablet_manager.cc:1398] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:15.942216 10304 ts_tablet_manager.cc:1398] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:15.942286 10763 consensus_queue.cc:260] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.942575 10763 ts_tablet_manager.cc:1398] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:15.964761 10760 tablet_bootstrap.cc:492] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d0f45862d7f14ce1976c1a37842f25ca: No bootstrap required, opened a new log
I20240502 14:01:15.964852 10760 ts_tablet_manager.cc:1367] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d0f45862d7f14ce1976c1a37842f25ca: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20240502 14:01:15.965298 10760 raft_consensus.cc:348] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.965409 10760 raft_consensus.cc:374] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.965430 10760 raft_consensus.cc:729] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.965575 10760 consensus_queue.cc:260] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d0f45862d7f14ce1976c1a37842f25ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.966073 10760 ts_tablet_manager.cc:1398] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d0f45862d7f14ce1976c1a37842f25ca: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:15.966259 10760 tablet_bootstrap.cc:492] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap starting.
I20240502 14:01:15.967056 10760 tablet_bootstrap.cc:654] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:15.968173 10760 tablet_bootstrap.cc:492] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca: No bootstrap required, opened a new log
I20240502 14:01:15.968276 10760 ts_tablet_manager.cc:1367] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:15.968703 10760 raft_consensus.cc:348] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.968832 10760 raft_consensus.cc:374] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:15.968878 10760 raft_consensus.cc:729] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Initialized, Role: FOLLOWER
I20240502 14:01:15.969004 10760 consensus_queue.cc:260] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.969269 10760 ts_tablet_manager.cc:1398] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:15.980823 10416 raft_consensus.cc:482] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:15.980923 10416 raft_consensus.cc:504] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.982219 10416 leader_election.cc:288] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749), 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:15.987298 9169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d4acfb937f4dd4b992c338fc6b3237" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" is_pre_election: true
I20240502 14:01:15.987442 9169 raft_consensus.cc:2454] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 0.
I20240502 14:01:15.987464 9302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d4acfb937f4dd4b992c338fc6b3237" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" is_pre_election: true
I20240502 14:01:15.987612 9302 raft_consensus.cc:2454] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 0.
I20240502 14:01:15.988078 9829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d4acfb937f4dd4b992c338fc6b3237" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b" is_pre_election: true
I20240502 14:01:15.988217 9829 raft_consensus.cc:2454] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 0.
I20240502 14:01:15.988680 9697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d4acfb937f4dd4b992c338fc6b3237" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288" is_pre_election: true
I20240502 14:01:15.988662 9499 leader_election.cc:302] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0488b0e4ed994db19d40aed8c15f945b, ccfb5bbc1bb54793812d11170ac2d25f, d0f45862d7f14ce1976c1a37842f25ca, d83af1bc823a4f2ca531f0d6825f8e16; no voters:
I20240502 14:01:15.988808 9697 raft_consensus.cc:2454] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 0.
I20240502 14:01:15.988909 10416 raft_consensus.cc:2781] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:15.988976 10416 raft_consensus.cc:482] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:15.989013 10416 raft_consensus.cc:3037] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.989998 10416 raft_consensus.cc:504] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.990602 10416 leader_election.cc:288] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [CANDIDATE]: Term 1 election: Requested vote from peers bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749), 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:15.990669 9169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d4acfb937f4dd4b992c338fc6b3237" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16"
I20240502 14:01:15.990780 9169 raft_consensus.cc:3037] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.990957 9697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d4acfb937f4dd4b992c338fc6b3237" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288"
I20240502 14:01:15.991068 9697 raft_consensus.cc:3037] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.991080 9829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d4acfb937f4dd4b992c338fc6b3237" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b"
I20240502 14:01:15.991173 9829 raft_consensus.cc:3037] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.991777 9169 raft_consensus.cc:2454] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 1.
I20240502 14:01:15.992064 9697 raft_consensus.cc:2454] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 1.
I20240502 14:01:15.992121 9829 raft_consensus.cc:2454] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 1.
I20240502 14:01:15.992724 9501 leader_election.cc:302] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0488b0e4ed994db19d40aed8c15f945b, d0f45862d7f14ce1976c1a37842f25ca, d83af1bc823a4f2ca531f0d6825f8e16, efa836edab8d4853a7ca47a23117e288; no voters:
I20240502 14:01:15.992790 9302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d4acfb937f4dd4b992c338fc6b3237" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f"
I20240502 14:01:15.992913 9302 raft_consensus.cc:3037] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.993165 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d4acfb937f4dd4b992c338fc6b3237" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" is_pre_election: true
I20240502 14:01:15.993218 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d4acfb937f4dd4b992c338fc6b3237" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d"
I20240502 14:01:15.993285 9038 raft_consensus.cc:2454] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 0.
I20240502 14:01:15.993561 9434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d4acfb937f4dd4b992c338fc6b3237" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de"
I20240502 14:01:15.993712 9434 raft_consensus.cc:3037] T 21d4acfb937f4dd4b992c338fc6b3237 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:15.993847 9302 raft_consensus.cc:2454] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 1.
I20240502 14:01:15.994510 9434 raft_consensus.cc:2454] T 21d4acfb937f4dd4b992c338fc6b3237 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 1.
I20240502 14:01:15.994551 10416 raft_consensus.cc:2781] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:15.993557 9433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d4acfb937f4dd4b992c338fc6b3237" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de" is_pre_election: true
I20240502 14:01:15.994714 9433 raft_consensus.cc:2362] T 21d4acfb937f4dd4b992c338fc6b3237 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 1. Re-sending same reply.
I20240502 14:01:15.994752 10416 raft_consensus.cc:686] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Becoming Leader. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Running, Role: LEADER
I20240502 14:01:15.995061 10416 consensus_queue.cc:237] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:15.997714 8890 catalog_manager.cc:5484] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca reported cstate change: term changed from 0 to 1, leader changed from <none> to d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69). New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.000051 10637 raft_consensus.cc:482] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:16.000136 10637 raft_consensus.cc:504] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:16.000747 10637 leader_election.cc:288] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:16.000893 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29eec0daa5d34e40a4ad82607b7ded90" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" is_pre_election: true
I20240502 14:01:16.001013 9564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29eec0daa5d34e40a4ad82607b7ded90" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca" is_pre_election: true
I20240502 14:01:16.001042 9036 raft_consensus.cc:2454] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 0.
I20240502 14:01:16.001121 9564 raft_consensus.cc:2454] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 0.
I20240502 14:01:16.001125 9829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29eec0daa5d34e40a4ad82607b7ded90" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b" is_pre_election: true
I20240502 14:01:16.001237 9829 raft_consensus.cc:2454] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 0.
I20240502 14:01:16.001351 9433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29eec0daa5d34e40a4ad82607b7ded90" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de" is_pre_election: true
I20240502 14:01:16.001466 9433 raft_consensus.cc:2454] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 0.
I20240502 14:01:16.001467 9697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29eec0daa5d34e40a4ad82607b7ded90" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288" is_pre_election: true
I20240502 14:01:16.001576 9697 raft_consensus.cc:2454] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 0.
I20240502 14:01:16.001667 9103 leader_election.cc:302] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0488b0e4ed994db19d40aed8c15f945b, bc9e4f2f668d48ea87d5aad4a2fb569d, d0f45862d7f14ce1976c1a37842f25ca, d83af1bc823a4f2ca531f0d6825f8e16; no voters:
I20240502 14:01:16.001812 10637 raft_consensus.cc:2781] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:16.001847 10637 raft_consensus.cc:482] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:16.001870 10637 raft_consensus.cc:3037] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.001966 9302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29eec0daa5d34e40a4ad82607b7ded90" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" is_pre_election: true
I20240502 14:01:16.002096 9302 raft_consensus.cc:2454] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 0.
I20240502 14:01:16.002897 10637 raft_consensus.cc:504] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:16.003485 10637 leader_election.cc:288] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [CANDIDATE]: Term 1 election: Requested vote from peers bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:16.003533 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29eec0daa5d34e40a4ad82607b7ded90" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d"
I20240502 14:01:16.003662 9036 raft_consensus.cc:3037] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.003758 9302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29eec0daa5d34e40a4ad82607b7ded90" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f"
I20240502 14:01:16.003867 9302 raft_consensus.cc:3037] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.003942 9433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29eec0daa5d34e40a4ad82607b7ded90" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de"
I20240502 14:01:16.004045 9433 raft_consensus.cc:3037] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.004776 9036 raft_consensus.cc:2454] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 1.
I20240502 14:01:16.004833 9302 raft_consensus.cc:2454] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 1.
I20240502 14:01:16.004971 9433 raft_consensus.cc:2454] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 1.
I20240502 14:01:16.005069 9564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29eec0daa5d34e40a4ad82607b7ded90" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca"
I20240502 14:01:16.005173 9564 raft_consensus.cc:3037] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.005429 9105 leader_election.cc:302] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1a490f170c614ee4900ef16c3287c7de, bc9e4f2f668d48ea87d5aad4a2fb569d, ccfb5bbc1bb54793812d11170ac2d25f, d83af1bc823a4f2ca531f0d6825f8e16; no voters:
I20240502 14:01:16.005465 9829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29eec0daa5d34e40a4ad82607b7ded90" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b"
I20240502 14:01:16.005489 9697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29eec0daa5d34e40a4ad82607b7ded90" candidate_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288"
I20240502 14:01:16.005556 9829 raft_consensus.cc:3037] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.005584 9697 raft_consensus.cc:3037] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.005723 10637 raft_consensus.cc:2781] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:16.005808 10637 raft_consensus.cc:686] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Becoming Leader. State: Replica: d83af1bc823a4f2ca531f0d6825f8e16, State: Running, Role: LEADER
I20240502 14:01:16.005939 10637 consensus_queue.cc:237] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:16.006091 9564 raft_consensus.cc:2454] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 1.
I20240502 14:01:16.006472 9697 raft_consensus.cc:2454] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 1.
I20240502 14:01:16.006541 9829 raft_consensus.cc:2454] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83af1bc823a4f2ca531f0d6825f8e16 in term 1.
I20240502 14:01:16.008445 8890 catalog_manager.cc:5484] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: term changed from 0 to 1, leader changed from <none> to d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66). New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.026070 10523 raft_consensus.cc:482] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:16.026168 10523 raft_consensus.cc:504] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:16.026744 10523 leader_election.cc:288] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907)
I20240502 14:01:16.027047 9169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97fa16722a1f4adcbf8a5dc3fdfab77a" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" is_pre_election: true
I20240502 14:01:16.027125 9697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97fa16722a1f4adcbf8a5dc3fdfab77a" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288" is_pre_election: true
I20240502 14:01:16.027165 9169 raft_consensus.cc:2454] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:16.027249 9697 raft_consensus.cc:2454] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:16.027540 9564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97fa16722a1f4adcbf8a5dc3fdfab77a" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca" is_pre_election: true
I20240502 14:01:16.027639 9564 raft_consensus.cc:2454] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:16.027706 9433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97fa16722a1f4adcbf8a5dc3fdfab77a" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de" is_pre_election: true
I20240502 14:01:16.027820 9433 raft_consensus.cc:2454] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:16.028144 9302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97fa16722a1f4adcbf8a5dc3fdfab77a" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" is_pre_election: true
I20240502 14:01:16.028266 9302 raft_consensus.cc:2454] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:16.028275 9764 leader_election.cc:302] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0488b0e4ed994db19d40aed8c15f945b, d0f45862d7f14ce1976c1a37842f25ca, d83af1bc823a4f2ca531f0d6825f8e16, efa836edab8d4853a7ca47a23117e288; no voters:
I20240502 14:01:16.028508 10523 raft_consensus.cc:2781] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:16.028553 10523 raft_consensus.cc:482] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:16.028589 10523 raft_consensus.cc:3037] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.028609 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97fa16722a1f4adcbf8a5dc3fdfab77a" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" is_pre_election: true
I20240502 14:01:16.028743 9036 raft_consensus.cc:2454] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:16.029543 10523 raft_consensus.cc:504] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:16.030068 10523 leader_election.cc:288] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 election: Requested vote from peers d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907)
I20240502 14:01:16.030225 9169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97fa16722a1f4adcbf8a5dc3fdfab77a" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16"
I20240502 14:01:16.030328 9169 raft_consensus.cc:3037] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.030474 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97fa16722a1f4adcbf8a5dc3fdfab77a" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d"
I20240502 14:01:16.030522 9564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97fa16722a1f4adcbf8a5dc3fdfab77a" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca"
I20240502 14:01:16.030575 9036 raft_consensus.cc:3037] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.030614 9564 raft_consensus.cc:3037] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.031257 9169 raft_consensus.cc:2454] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:16.031414 9564 raft_consensus.cc:2454] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:16.031492 9036 raft_consensus.cc:2454] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:16.031605 9697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97fa16722a1f4adcbf8a5dc3fdfab77a" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288"
I20240502 14:01:16.031701 9697 raft_consensus.cc:3037] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.031853 9302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97fa16722a1f4adcbf8a5dc3fdfab77a" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f"
I20240502 14:01:16.031948 9302 raft_consensus.cc:3037] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.031944 9433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97fa16722a1f4adcbf8a5dc3fdfab77a" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de"
I20240502 14:01:16.032038 9433 raft_consensus.cc:3037] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.032599 9697 raft_consensus.cc:2454] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:16.032786 9302 raft_consensus.cc:2454] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:16.032864 9766 leader_election.cc:302] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0488b0e4ed994db19d40aed8c15f945b, bc9e4f2f668d48ea87d5aad4a2fb569d, d0f45862d7f14ce1976c1a37842f25ca, efa836edab8d4853a7ca47a23117e288; no voters:
I20240502 14:01:16.033015 9433 raft_consensus.cc:2454] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:16.033059 10523 raft_consensus.cc:2781] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:16.033123 10523 raft_consensus.cc:686] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Becoming Leader. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Running, Role: LEADER
I20240502 14:01:16.033279 10523 consensus_queue.cc:237] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:16.035614 8890 catalog_manager.cc:5484] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71). New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } }
W20240502 14:01:16.159474 10795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:16.159696 10795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:16.163051 10795 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20240502 14:01:16.250577 9433 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.251283 9564 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.251353 10636 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.252030 10636 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.256526 9697 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.257040 10791 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.257318 9434 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.258152 9302 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.258468 9829 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.259379 9036 raft_consensus.cc:3037] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.260735 9038 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.262599 9829 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.263108 9302 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.263450 10579 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.263741 10579 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.263955 10579 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.268503 9697 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.269177 10416 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.269479 10416 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.269693 10416 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.272395 10635 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.277977 9169 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.279918 9037 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.280396 10718 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.283711 10717 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.284479 9301 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.284922 10717 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.288095 9697 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.288465 10717 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.289666 9433 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.292830 9564 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.293740 9036 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.294286 10791 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.295686 10712 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.296551 10734 mvcc.cc:205] Tried to move back new op lower bound from 7023241118717702144 to 7023241117725257728. Current Snapshot: MvccSnapshot[applied={T|T < 7023241118717702144}]
I20240502 14:01:16.301024 10523 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.305276 9168 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.306226 10791 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 222.245 ms
time per row: 0.111123 ms
I20240502 14:01:16.430728 8896 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46030:
name: "rebalance_test_table_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240502 14:01:16.431257 8896 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_5 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240502 14:01:16.436733 9414 tablet_service.cc:1467] Processing CreateTablet for tablet ded9a2cda7b142d79489c8541e8e15f7 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.437052 9414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ded9a2cda7b142d79489c8541e8e15f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.437178 9017 tablet_service.cc:1467] Processing CreateTablet for tablet ded9a2cda7b142d79489c8541e8e15f7 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.437481 9017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ded9a2cda7b142d79489c8541e8e15f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.439316 10759 tablet_bootstrap.cc:492] T ded9a2cda7b142d79489c8541e8e15f7 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:16.440023 9545 tablet_service.cc:1467] Processing CreateTablet for tablet ded9a2cda7b142d79489c8541e8e15f7 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.440207 10759 tablet_bootstrap.cc:654] T ded9a2cda7b142d79489c8541e8e15f7 P 1a490f170c614ee4900ef16c3287c7de: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.440274 9545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ded9a2cda7b142d79489c8541e8e15f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.440850 10304 tablet_bootstrap.cc:492] T ded9a2cda7b142d79489c8541e8e15f7 P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap starting.
I20240502 14:01:16.441669 10304 tablet_bootstrap.cc:654] T ded9a2cda7b142d79489c8541e8e15f7 P bc9e4f2f668d48ea87d5aad4a2fb569d: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.441620 9544 tablet_service.cc:1467] Processing CreateTablet for tablet 8c10fb9748c747c6a89066fa113cda3c (DEFAULT_TABLE table=rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.441869 9544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c10fb9748c747c6a89066fa113cda3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.442591 10760 tablet_bootstrap.cc:492] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap starting.
I20240502 14:01:16.443267 10304 tablet_bootstrap.cc:492] T ded9a2cda7b142d79489c8541e8e15f7 P bc9e4f2f668d48ea87d5aad4a2fb569d: No bootstrap required, opened a new log
I20240502 14:01:16.443347 10304 ts_tablet_manager.cc:1367] T ded9a2cda7b142d79489c8541e8e15f7 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 14:01:16.443361 10760 tablet_bootstrap.cc:654] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.443802 10304 raft_consensus.cc:348] T ded9a2cda7b142d79489c8541e8e15f7 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.443917 10304 raft_consensus.cc:374] T ded9a2cda7b142d79489c8541e8e15f7 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.443953 10304 raft_consensus.cc:729] T ded9a2cda7b142d79489c8541e8e15f7 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9e4f2f668d48ea87d5aad4a2fb569d, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.443933 9546 tablet_service.cc:1467] Processing CreateTablet for tablet 5f84e791335948e180f5575f8174efd1 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.444136 9546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f84e791335948e180f5575f8174efd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.444078 10304 consensus_queue.cc:260] T ded9a2cda7b142d79489c8541e8e15f7 P bc9e4f2f668d48ea87d5aad4a2fb569d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.444403 10304 ts_tablet_manager.cc:1398] T ded9a2cda7b142d79489c8541e8e15f7 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:16.444638 9678 tablet_service.cc:1467] Processing CreateTablet for tablet ded9a2cda7b142d79489c8541e8e15f7 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.444900 9678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ded9a2cda7b142d79489c8541e8e15f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.446005 10759 tablet_bootstrap.cc:492] T ded9a2cda7b142d79489c8541e8e15f7 P 1a490f170c614ee4900ef16c3287c7de: No bootstrap required, opened a new log
I20240502 14:01:16.446108 10759 ts_tablet_manager.cc:1367] T ded9a2cda7b142d79489c8541e8e15f7 P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:01:16.446524 10759 raft_consensus.cc:348] T ded9a2cda7b142d79489c8541e8e15f7 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.446640 10759 raft_consensus.cc:374] T ded9a2cda7b142d79489c8541e8e15f7 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.446681 10759 raft_consensus.cc:729] T ded9a2cda7b142d79489c8541e8e15f7 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.446802 10759 consensus_queue.cc:260] T ded9a2cda7b142d79489c8541e8e15f7 P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.447042 10853 tablet_bootstrap.cc:492] T ded9a2cda7b142d79489c8541e8e15f7 P efa836edab8d4853a7ca47a23117e288: Bootstrap starting.
I20240502 14:01:16.447093 10759 ts_tablet_manager.cc:1398] T ded9a2cda7b142d79489c8541e8e15f7 P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:16.447207 9676 tablet_service.cc:1467] Processing CreateTablet for tablet 8c10fb9748c747c6a89066fa113cda3c (DEFAULT_TABLE table=rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.447448 9676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c10fb9748c747c6a89066fa113cda3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.448117 10853 tablet_bootstrap.cc:654] T ded9a2cda7b142d79489c8541e8e15f7 P efa836edab8d4853a7ca47a23117e288: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.449581 9677 tablet_service.cc:1467] Processing CreateTablet for tablet 5f84e791335948e180f5575f8174efd1 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.449805 9677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f84e791335948e180f5575f8174efd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.451583 9282 tablet_service.cc:1467] Processing CreateTablet for tablet ded9a2cda7b142d79489c8541e8e15f7 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.451848 9282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ded9a2cda7b142d79489c8541e8e15f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.453027 10853 tablet_bootstrap.cc:492] T ded9a2cda7b142d79489c8541e8e15f7 P efa836edab8d4853a7ca47a23117e288: No bootstrap required, opened a new log
I20240502 14:01:16.453102 10853 ts_tablet_manager.cc:1367] T ded9a2cda7b142d79489c8541e8e15f7 P efa836edab8d4853a7ca47a23117e288: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20240502 14:01:16.453442 10853 raft_consensus.cc:348] T ded9a2cda7b142d79489c8541e8e15f7 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.453552 10853 raft_consensus.cc:374] T ded9a2cda7b142d79489c8541e8e15f7 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.453577 10853 raft_consensus.cc:729] T ded9a2cda7b142d79489c8541e8e15f7 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa836edab8d4853a7ca47a23117e288, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.453680 10853 consensus_queue.cc:260] T ded9a2cda7b142d79489c8541e8e15f7 P efa836edab8d4853a7ca47a23117e288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.453727 9281 tablet_service.cc:1467] Processing CreateTablet for tablet 8c10fb9748c747c6a89066fa113cda3c (DEFAULT_TABLE table=rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.453912 10853 ts_tablet_manager.cc:1398] T ded9a2cda7b142d79489c8541e8e15f7 P efa836edab8d4853a7ca47a23117e288: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:16.453938 9281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c10fb9748c747c6a89066fa113cda3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.454036 10853 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288: Bootstrap starting.
I20240502 14:01:16.454275 9280 tablet_service.cc:1467] Processing CreateTablet for tablet 5f84e791335948e180f5575f8174efd1 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.455385 10760 tablet_bootstrap.cc:492] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca: No bootstrap required, opened a new log
I20240502 14:01:16.455471 10760 ts_tablet_manager.cc:1367] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240502 14:01:16.455960 10760 raft_consensus.cc:348] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.456077 10760 raft_consensus.cc:374] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.456108 10760 raft_consensus.cc:729] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.456218 10760 consensus_queue.cc:260] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.456694 10760 ts_tablet_manager.cc:1398] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:16.456848 10760 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap starting.
I20240502 14:01:16.457746 10760 tablet_bootstrap.cc:654] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.459278 9412 tablet_service.cc:1467] Processing CreateTablet for tablet 5f84e791335948e180f5575f8174efd1 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.459404 9017 tablet_service.cc:1467] Processing CreateTablet for tablet 8c10fb9748c747c6a89066fa113cda3c (DEFAULT_TABLE table=rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.459517 9412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f84e791335948e180f5575f8174efd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.459585 9414 tablet_service.cc:1467] Processing CreateTablet for tablet 8c10fb9748c747c6a89066fa113cda3c (DEFAULT_TABLE table=rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.459635 9017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c10fb9748c747c6a89066fa113cda3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.459769 9414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c10fb9748c747c6a89066fa113cda3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.461601 10759 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:16.461802 9018 tablet_service.cc:1467] Processing CreateTablet for tablet 5f84e791335948e180f5575f8174efd1 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.461973 9018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f84e791335948e180f5575f8174efd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.462525 10759 tablet_bootstrap.cc:654] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.463534 9280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f84e791335948e180f5575f8174efd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.463646 10759 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de: No bootstrap required, opened a new log
I20240502 14:01:16.463713 10759 ts_tablet_manager.cc:1367] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 14:01:16.463923 9150 tablet_service.cc:1467] Processing CreateTablet for tablet 8c10fb9748c747c6a89066fa113cda3c (DEFAULT_TABLE table=rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.463923 9148 tablet_service.cc:1467] Processing CreateTablet for tablet ded9a2cda7b142d79489c8541e8e15f7 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.464089 10759 raft_consensus.cc:348] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.464196 10759 raft_consensus.cc:374] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.464242 10759 raft_consensus.cc:729] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.464251 9148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ded9a2cda7b142d79489c8541e8e15f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.464391 10759 consensus_queue.cc:260] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.464489 9149 tablet_service.cc:1467] Processing CreateTablet for tablet 5f84e791335948e180f5575f8174efd1 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.464722 9149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f84e791335948e180f5575f8174efd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.465178 10759 ts_tablet_manager.cc:1398] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:16.465342 10759 tablet_bootstrap.cc:492] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:16.467628 10759 tablet_bootstrap.cc:654] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.467664 9150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c10fb9748c747c6a89066fa113cda3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.469208 10759 tablet_bootstrap.cc:492] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de: No bootstrap required, opened a new log
I20240502 14:01:16.469293 10759 ts_tablet_manager.cc:1367] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:01:16.469496 10853 tablet_bootstrap.cc:654] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.469765 10759 raft_consensus.cc:348] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.469888 10759 raft_consensus.cc:374] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.469918 10759 raft_consensus.cc:729] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.470036 10759 consensus_queue.cc:260] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.470672 10759 ts_tablet_manager.cc:1398] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:16.471139 10304 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap starting.
I20240502 14:01:16.472028 10304 tablet_bootstrap.cc:654] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.473206 10853 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288: No bootstrap required, opened a new log
I20240502 14:01:16.473318 10853 ts_tablet_manager.cc:1367] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.002s
I20240502 14:01:16.473348 10304 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d: No bootstrap required, opened a new log
I20240502 14:01:16.473423 10304 ts_tablet_manager.cc:1367] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 14:01:16.473559 9810 tablet_service.cc:1467] Processing CreateTablet for tablet ded9a2cda7b142d79489c8541e8e15f7 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.473820 9810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ded9a2cda7b142d79489c8541e8e15f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.473750 10853 raft_consensus.cc:348] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.473866 10853 raft_consensus.cc:374] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.473899 10853 raft_consensus.cc:729] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa836edab8d4853a7ca47a23117e288, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.473837 10304 raft_consensus.cc:348] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.473863 9808 tablet_service.cc:1467] Processing CreateTablet for tablet 8c10fb9748c747c6a89066fa113cda3c (DEFAULT_TABLE table=rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.473939 10304 raft_consensus.cc:374] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.473970 10304 raft_consensus.cc:729] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9e4f2f668d48ea87d5aad4a2fb569d, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.474097 9808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c10fb9748c747c6a89066fa113cda3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.474093 10304 consensus_queue.cc:260] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.474357 10304 ts_tablet_manager.cc:1398] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:16.474304 10853 consensus_queue.cc:260] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.474509 10304 tablet_bootstrap.cc:492] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap starting.
I20240502 14:01:16.475307 10853 ts_tablet_manager.cc:1398] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:01:16.475350 10304 tablet_bootstrap.cc:654] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.475459 10853 tablet_bootstrap.cc:492] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288: Bootstrap starting.
I20240502 14:01:16.476099 9809 tablet_service.cc:1467] Processing CreateTablet for tablet 5f84e791335948e180f5575f8174efd1 (DEFAULT_TABLE table=rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.476289 10853 tablet_bootstrap.cc:654] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.476357 9809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f84e791335948e180f5575f8174efd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.476760 10304 tablet_bootstrap.cc:492] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d: No bootstrap required, opened a new log
I20240502 14:01:16.476828 10304 ts_tablet_manager.cc:1367] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 14:01:16.477159 10304 raft_consensus.cc:348] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.477260 10304 raft_consensus.cc:374] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.477283 10304 raft_consensus.cc:729] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9e4f2f668d48ea87d5aad4a2fb569d, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.477421 10304 consensus_queue.cc:260] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.477663 10304 ts_tablet_manager.cc:1398] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:16.477896 10858 tablet_bootstrap.cc:492] T ded9a2cda7b142d79489c8541e8e15f7 P d83af1bc823a4f2ca531f0d6825f8e16: Bootstrap starting.
I20240502 14:01:16.478673 10858 tablet_bootstrap.cc:654] T ded9a2cda7b142d79489c8541e8e15f7 P d83af1bc823a4f2ca531f0d6825f8e16: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.479133 10862 tablet_bootstrap.cc:492] T ded9a2cda7b142d79489c8541e8e15f7 P 0488b0e4ed994db19d40aed8c15f945b: Bootstrap starting.
I20240502 14:01:16.479974 10862 tablet_bootstrap.cc:654] T ded9a2cda7b142d79489c8541e8e15f7 P 0488b0e4ed994db19d40aed8c15f945b: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.481148 10857 tablet_bootstrap.cc:492] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f: Bootstrap starting.
I20240502 14:01:16.481518 10862 tablet_bootstrap.cc:492] T ded9a2cda7b142d79489c8541e8e15f7 P 0488b0e4ed994db19d40aed8c15f945b: No bootstrap required, opened a new log
I20240502 14:01:16.481607 10862 ts_tablet_manager.cc:1367] T ded9a2cda7b142d79489c8541e8e15f7 P 0488b0e4ed994db19d40aed8c15f945b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:01:16.481926 10857 tablet_bootstrap.cc:654] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.481952 10862 raft_consensus.cc:348] T ded9a2cda7b142d79489c8541e8e15f7 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.482081 10862 raft_consensus.cc:374] T ded9a2cda7b142d79489c8541e8e15f7 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.482153 10862 raft_consensus.cc:729] T ded9a2cda7b142d79489c8541e8e15f7 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.482300 10862 consensus_queue.cc:260] T ded9a2cda7b142d79489c8541e8e15f7 P 0488b0e4ed994db19d40aed8c15f945b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.482601 10862 ts_tablet_manager.cc:1398] T ded9a2cda7b142d79489c8541e8e15f7 P 0488b0e4ed994db19d40aed8c15f945b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:16.482992 10858 tablet_bootstrap.cc:492] T ded9a2cda7b142d79489c8541e8e15f7 P d83af1bc823a4f2ca531f0d6825f8e16: No bootstrap required, opened a new log
I20240502 14:01:16.483014 10862 tablet_bootstrap.cc:492] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b: Bootstrap starting.
I20240502 14:01:16.483085 10858 ts_tablet_manager.cc:1367] T ded9a2cda7b142d79489c8541e8e15f7 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 14:01:16.483441 10858 raft_consensus.cc:348] T ded9a2cda7b142d79489c8541e8e15f7 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.483569 10858 raft_consensus.cc:374] T ded9a2cda7b142d79489c8541e8e15f7 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.483604 10858 raft_consensus.cc:729] T ded9a2cda7b142d79489c8541e8e15f7 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83af1bc823a4f2ca531f0d6825f8e16, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.483830 10858 consensus_queue.cc:260] T ded9a2cda7b142d79489c8541e8e15f7 P d83af1bc823a4f2ca531f0d6825f8e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.483891 10862 tablet_bootstrap.cc:654] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.484177 10858 ts_tablet_manager.cc:1398] T ded9a2cda7b142d79489c8541e8e15f7 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:16.484380 10858 tablet_bootstrap.cc:492] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16: Bootstrap starting.
I20240502 14:01:16.484690 10857 tablet_bootstrap.cc:492] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f: No bootstrap required, opened a new log
I20240502 14:01:16.484784 10857 ts_tablet_manager.cc:1367] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 14:01:16.484791 10853 tablet_bootstrap.cc:492] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288: No bootstrap required, opened a new log
I20240502 14:01:16.484867 10853 ts_tablet_manager.cc:1367] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240502 14:01:16.485255 10857 raft_consensus.cc:348] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.485280 10853 raft_consensus.cc:348] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.485386 10857 raft_consensus.cc:374] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.485386 10853 raft_consensus.cc:374] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.485421 10853 raft_consensus.cc:729] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa836edab8d4853a7ca47a23117e288, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.485421 10857 raft_consensus.cc:729] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.485553 10857 consensus_queue.cc:260] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.485848 10857 ts_tablet_manager.cc:1398] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:16.486007 10857 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f: Bootstrap starting.
I20240502 14:01:16.485965 10853 consensus_queue.cc:260] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.486696 10853 ts_tablet_manager.cc:1398] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:16.486835 10857 tablet_bootstrap.cc:654] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.487314 10862 tablet_bootstrap.cc:492] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b: No bootstrap required, opened a new log
I20240502 14:01:16.487414 10862 ts_tablet_manager.cc:1367] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 14:01:16.487807 10862 raft_consensus.cc:348] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.487927 10862 raft_consensus.cc:374] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.487959 10862 raft_consensus.cc:729] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.488066 10862 consensus_queue.cc:260] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.488281 10858 tablet_bootstrap.cc:654] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.488392 10862 ts_tablet_manager.cc:1398] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:16.488552 10862 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b: Bootstrap starting.
I20240502 14:01:16.489179 10858 tablet_bootstrap.cc:492] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16: No bootstrap required, opened a new log
I20240502 14:01:16.489244 10858 ts_tablet_manager.cc:1367] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:01:16.489416 10862 tablet_bootstrap.cc:654] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.489617 10858 raft_consensus.cc:348] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.489715 10858 raft_consensus.cc:374] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.489739 10858 raft_consensus.cc:729] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83af1bc823a4f2ca531f0d6825f8e16, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.489859 10858 consensus_queue.cc:260] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.490190 10858 ts_tablet_manager.cc:1398] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:16.490351 10858 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16: Bootstrap starting.
I20240502 14:01:16.490533 10760 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca: No bootstrap required, opened a new log
I20240502 14:01:16.490620 10760 ts_tablet_manager.cc:1367] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.000s
I20240502 14:01:16.491155 10858 tablet_bootstrap.cc:654] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.491041 10760 raft_consensus.cc:348] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.491207 10760 raft_consensus.cc:374] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.491261 10760 raft_consensus.cc:729] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.491422 10760 consensus_queue.cc:260] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.491742 10760 ts_tablet_manager.cc:1398] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:16.491901 10760 tablet_bootstrap.cc:492] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap starting.
I20240502 14:01:16.492738 10760 tablet_bootstrap.cc:654] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.492969 10857 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f: No bootstrap required, opened a new log
I20240502 14:01:16.493048 10857 ts_tablet_manager.cc:1367] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 14:01:16.493144 10858 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16: No bootstrap required, opened a new log
I20240502 14:01:16.493234 10858 ts_tablet_manager.cc:1367] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 14:01:16.493445 10857 raft_consensus.cc:348] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.493587 10857 raft_consensus.cc:374] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.493619 10857 raft_consensus.cc:729] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.493630 10858 raft_consensus.cc:348] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.493731 10858 raft_consensus.cc:374] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.493741 10862 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b: No bootstrap required, opened a new log
I20240502 14:01:16.493760 10858 raft_consensus.cc:729] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83af1bc823a4f2ca531f0d6825f8e16, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.493727 10857 consensus_queue.cc:260] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.493803 10862 ts_tablet_manager.cc:1367] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 14:01:16.493866 10858 consensus_queue.cc:260] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.494005 10857 ts_tablet_manager.cc:1398] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:16.493980 10398 raft_consensus.cc:482] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:16.494074 10398 raft_consensus.cc:504] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.494145 10858 ts_tablet_manager.cc:1398] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:16.494169 10857 tablet_bootstrap.cc:492] T ded9a2cda7b142d79489c8541e8e15f7 P ccfb5bbc1bb54793812d11170ac2d25f: Bootstrap starting.
I20240502 14:01:16.494184 10862 raft_consensus.cc:348] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.494285 10862 raft_consensus.cc:374] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.494315 10862 raft_consensus.cc:729] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.494727 10862 consensus_queue.cc:260] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.494802 10398 leader_election.cc:288] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679), bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:16.494869 10760 tablet_bootstrap.cc:492] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca: No bootstrap required, opened a new log
I20240502 14:01:16.494946 10760 ts_tablet_manager.cc:1367] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:01:16.495019 10857 tablet_bootstrap.cc:654] T ded9a2cda7b142d79489c8541e8e15f7 P ccfb5bbc1bb54793812d11170ac2d25f: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.495347 10760 raft_consensus.cc:348] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.495460 10760 raft_consensus.cc:374] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.495501 10760 raft_consensus.cc:729] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.495678 9828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f84e791335948e180f5575f8174efd1" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b" is_pre_election: true
I20240502 14:01:16.495784 9828 raft_consensus.cc:2454] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:16.496021 10862 ts_tablet_manager.cc:1398] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:16.496091 9300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f84e791335948e180f5575f8174efd1" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" is_pre_election: true
I20240502 14:01:16.496210 9300 raft_consensus.cc:2454] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:16.496166 10760 consensus_queue.cc:260] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.496513 10760 ts_tablet_manager.cc:1398] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:16.496795 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f84e791335948e180f5575f8174efd1" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" is_pre_election: true
I20240502 14:01:16.496925 9038 raft_consensus.cc:2454] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:16.497135 9168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f84e791335948e180f5575f8174efd1" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" is_pre_election: true
I20240502 14:01:16.497273 9168 raft_consensus.cc:2454] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:16.497603 9369 leader_election.cc:302] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0488b0e4ed994db19d40aed8c15f945b, 1a490f170c614ee4900ef16c3287c7de, bc9e4f2f668d48ea87d5aad4a2fb569d, d83af1bc823a4f2ca531f0d6825f8e16; no voters:
I20240502 14:01:16.497762 10838 raft_consensus.cc:482] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:16.497856 10838 raft_consensus.cc:504] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.497886 9564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f84e791335948e180f5575f8174efd1" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca" is_pre_election: true
I20240502 14:01:16.498008 9564 raft_consensus.cc:2454] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:16.498275 10398 raft_consensus.cc:2781] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:16.498337 10398 raft_consensus.cc:482] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:16.498320 9697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f84e791335948e180f5575f8174efd1" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288" is_pre_election: true
I20240502 14:01:16.498378 10398 raft_consensus.cc:3037] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.498440 9697 raft_consensus.cc:2454] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:16.498503 10838 leader_election.cc:288] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679), bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:16.498713 9564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c10fb9748c747c6a89066fa113cda3c" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca" is_pre_election: true
I20240502 14:01:16.498809 9564 raft_consensus.cc:2454] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:16.498962 9697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c10fb9748c747c6a89066fa113cda3c" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288" is_pre_election: true
I20240502 14:01:16.499053 9697 raft_consensus.cc:2454] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:16.499399 10857 tablet_bootstrap.cc:492] T ded9a2cda7b142d79489c8541e8e15f7 P ccfb5bbc1bb54793812d11170ac2d25f: No bootstrap required, opened a new log
I20240502 14:01:16.499470 10857 ts_tablet_manager.cc:1367] T ded9a2cda7b142d79489c8541e8e15f7 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 14:01:16.499408 10398 raft_consensus.cc:504] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.499976 10398 leader_election.cc:288] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [CANDIDATE]: Term 1 election: Requested vote from peers d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679), bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:16.500054 9566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f84e791335948e180f5575f8174efd1" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca"
I20240502 14:01:16.500149 9566 raft_consensus.cc:3037] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.500180 9828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c10fb9748c747c6a89066fa113cda3c" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b" is_pre_election: true
I20240502 14:01:16.500281 9828 raft_consensus.cc:2454] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:16.500420 9829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f84e791335948e180f5575f8174efd1" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b"
I20240502 14:01:16.500507 9829 raft_consensus.cc:3037] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.500542 9302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f84e791335948e180f5575f8174efd1" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f"
I20240502 14:01:16.500653 9302 raft_consensus.cc:3037] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.501087 9566 raft_consensus.cc:2454] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:16.501300 9036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f84e791335948e180f5575f8174efd1" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d"
I20240502 14:01:16.501423 9036 raft_consensus.cc:3037] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.501422 9829 raft_consensus.cc:2454] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:16.501410 9168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c10fb9748c747c6a89066fa113cda3c" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" is_pre_election: true
I20240502 14:01:16.501538 9168 raft_consensus.cc:2454] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:16.501606 9302 raft_consensus.cc:2454] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:16.501632 9169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f84e791335948e180f5575f8174efd1" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16"
I20240502 14:01:16.501720 9169 raft_consensus.cc:3037] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.500550 9300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c10fb9748c747c6a89066fa113cda3c" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" is_pre_election: true
I20240502 14:01:16.501830 9300 raft_consensus.cc:2454] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:16.502329 9036 raft_consensus.cc:2454] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:16.502517 9367 leader_election.cc:302] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1a490f170c614ee4900ef16c3287c7de, ccfb5bbc1bb54793812d11170ac2d25f, d0f45862d7f14ce1976c1a37842f25ca, efa836edab8d4853a7ca47a23117e288; no voters:
I20240502 14:01:16.502656 9169 raft_consensus.cc:2454] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:16.502682 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c10fb9748c747c6a89066fa113cda3c" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" is_pre_election: true
I20240502 14:01:16.502779 9038 raft_consensus.cc:2454] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 0.
I20240502 14:01:16.500232 9697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f84e791335948e180f5575f8174efd1" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288"
I20240502 14:01:16.502933 9697 raft_consensus.cc:3037] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.502951 9369 leader_election.cc:302] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0488b0e4ed994db19d40aed8c15f945b, 1a490f170c614ee4900ef16c3287c7de, ccfb5bbc1bb54793812d11170ac2d25f, d83af1bc823a4f2ca531f0d6825f8e16; no voters:
I20240502 14:01:16.503389 10838 raft_consensus.cc:2781] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:16.503449 10838 raft_consensus.cc:482] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:16.503496 10838 raft_consensus.cc:3037] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.503567 10398 raft_consensus.cc:2781] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:16.503633 10398 raft_consensus.cc:686] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Becoming Leader. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Running, Role: LEADER
I20240502 14:01:16.503772 10398 consensus_queue.cc:237] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.503921 9697 raft_consensus.cc:2454] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:16.504354 10857 raft_consensus.cc:348] T ded9a2cda7b142d79489c8541e8e15f7 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.504462 10857 raft_consensus.cc:374] T ded9a2cda7b142d79489c8541e8e15f7 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.504505 10857 raft_consensus.cc:729] T ded9a2cda7b142d79489c8541e8e15f7 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.504458 10838 raft_consensus.cc:504] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.504968 10838 leader_election.cc:288] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [CANDIDATE]: Term 1 election: Requested vote from peers efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679), bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:16.505347 9564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c10fb9748c747c6a89066fa113cda3c" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca"
I20240502 14:01:16.505447 9564 raft_consensus.cc:3037] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.506054 9300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c10fb9748c747c6a89066fa113cda3c" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f"
I20240502 14:01:16.506160 9300 raft_consensus.cc:3037] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.506285 9697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c10fb9748c747c6a89066fa113cda3c" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288"
I20240502 14:01:16.506382 9697 raft_consensus.cc:3037] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.506426 9564 raft_consensus.cc:2454] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:16.504935 10857 consensus_queue.cc:260] T ded9a2cda7b142d79489c8541e8e15f7 P ccfb5bbc1bb54793812d11170ac2d25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.507016 9300 raft_consensus.cc:2454] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:16.507230 9697 raft_consensus.cc:2454] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:16.507587 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c10fb9748c747c6a89066fa113cda3c" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d"
I20240502 14:01:16.507599 9829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c10fb9748c747c6a89066fa113cda3c" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b"
I20240502 14:01:16.507707 9829 raft_consensus.cc:3037] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.507706 9038 raft_consensus.cc:3037] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.507726 9368 leader_election.cc:302] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1a490f170c614ee4900ef16c3287c7de, ccfb5bbc1bb54793812d11170ac2d25f, d0f45862d7f14ce1976c1a37842f25ca, efa836edab8d4853a7ca47a23117e288; no voters:
I20240502 14:01:16.507885 10838 raft_consensus.cc:2781] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:16.507946 9169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c10fb9748c747c6a89066fa113cda3c" candidate_uuid: "1a490f170c614ee4900ef16c3287c7de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16"
I20240502 14:01:16.508008 10838 raft_consensus.cc:686] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Becoming Leader. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Running, Role: LEADER
I20240502 14:01:16.508035 9169 raft_consensus.cc:3037] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.508129 10838 consensus_queue.cc:237] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.508699 9038 raft_consensus.cc:2454] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:16.508728 9829 raft_consensus.cc:2454] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:16.508999 9169 raft_consensus.cc:2454] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a490f170c614ee4900ef16c3287c7de in term 1.
I20240502 14:01:16.509717 10857 ts_tablet_manager.cc:1398] T ded9a2cda7b142d79489c8541e8e15f7 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent starting tablet: real 0.010s user 0.001s sys 0.000s
I20240502 14:01:16.511258 8889 catalog_manager.cc:5484] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a490f170c614ee4900ef16c3287c7de (127.24.66.68). New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.514307 8889 catalog_manager.cc:5484] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a490f170c614ee4900ef16c3287c7de (127.24.66.68). New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.524509 10416 raft_consensus.cc:482] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:16.524631 10416 raft_consensus.cc:504] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.525206 10416 leader_election.cc:288] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679), bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:16.525435 9697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ded9a2cda7b142d79489c8541e8e15f7" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288" is_pre_election: true
I20240502 14:01:16.525518 9433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ded9a2cda7b142d79489c8541e8e15f7" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de" is_pre_election: true
I20240502 14:01:16.525574 9697 raft_consensus.cc:2454] T ded9a2cda7b142d79489c8541e8e15f7 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 0.
I20240502 14:01:16.525619 9433 raft_consensus.cc:2454] T ded9a2cda7b142d79489c8541e8e15f7 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 0.
I20240502 14:01:16.525607 9169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ded9a2cda7b142d79489c8541e8e15f7" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" is_pre_election: true
I20240502 14:01:16.525727 9169 raft_consensus.cc:2454] T ded9a2cda7b142d79489c8541e8e15f7 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 0.
I20240502 14:01:16.526074 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ded9a2cda7b142d79489c8541e8e15f7" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" is_pre_election: true
I20240502 14:01:16.526207 9038 raft_consensus.cc:2454] T ded9a2cda7b142d79489c8541e8e15f7 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 0.
I20240502 14:01:16.526191 9829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ded9a2cda7b142d79489c8541e8e15f7" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b" is_pre_election: true
I20240502 14:01:16.526284 9300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ded9a2cda7b142d79489c8541e8e15f7" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" is_pre_election: true
I20240502 14:01:16.526329 9829 raft_consensus.cc:2454] T ded9a2cda7b142d79489c8541e8e15f7 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 0.
I20240502 14:01:16.526393 9300 raft_consensus.cc:2454] T ded9a2cda7b142d79489c8541e8e15f7 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 0.
I20240502 14:01:16.526485 9501 leader_election.cc:302] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1a490f170c614ee4900ef16c3287c7de, bc9e4f2f668d48ea87d5aad4a2fb569d, d0f45862d7f14ce1976c1a37842f25ca, efa836edab8d4853a7ca47a23117e288; no voters:
I20240502 14:01:16.526672 10416 raft_consensus.cc:2781] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:16.526731 10416 raft_consensus.cc:482] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:16.526780 10416 raft_consensus.cc:3037] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.527631 10416 raft_consensus.cc:504] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.528225 10416 leader_election.cc:288] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [CANDIDATE]: Term 1 election: Requested vote from peers 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679), bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:16.528501 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ded9a2cda7b142d79489c8541e8e15f7" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d"
I20240502 14:01:16.528501 9169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ded9a2cda7b142d79489c8541e8e15f7" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16"
I20240502 14:01:16.528507 9697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ded9a2cda7b142d79489c8541e8e15f7" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288"
I20240502 14:01:16.528617 9169 raft_consensus.cc:3037] T ded9a2cda7b142d79489c8541e8e15f7 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.528617 9038 raft_consensus.cc:3037] T ded9a2cda7b142d79489c8541e8e15f7 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.528623 9697 raft_consensus.cc:3037] T ded9a2cda7b142d79489c8541e8e15f7 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.528719 9829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ded9a2cda7b142d79489c8541e8e15f7" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b"
I20240502 14:01:16.528810 9829 raft_consensus.cc:3037] T ded9a2cda7b142d79489c8541e8e15f7 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.529490 9038 raft_consensus.cc:2454] T ded9a2cda7b142d79489c8541e8e15f7 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 1.
I20240502 14:01:16.529490 9169 raft_consensus.cc:2454] T ded9a2cda7b142d79489c8541e8e15f7 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 1.
I20240502 14:01:16.529534 9697 raft_consensus.cc:2454] T ded9a2cda7b142d79489c8541e8e15f7 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 1.
I20240502 14:01:16.529712 9829 raft_consensus.cc:2454] T ded9a2cda7b142d79489c8541e8e15f7 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 1.
I20240502 14:01:16.529837 9502 leader_election.cc:302] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: bc9e4f2f668d48ea87d5aad4a2fb569d, d0f45862d7f14ce1976c1a37842f25ca, d83af1bc823a4f2ca531f0d6825f8e16, efa836edab8d4853a7ca47a23117e288; no voters:
I20240502 14:01:16.530169 10416 raft_consensus.cc:2781] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:16.530189 9300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ded9a2cda7b142d79489c8541e8e15f7" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f"
I20240502 14:01:16.530182 9433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ded9a2cda7b142d79489c8541e8e15f7" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de"
I20240502 14:01:16.530254 10416 raft_consensus.cc:686] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Becoming Leader. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Running, Role: LEADER
I20240502 14:01:16.530305 9300 raft_consensus.cc:3037] T ded9a2cda7b142d79489c8541e8e15f7 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.530340 9433 raft_consensus.cc:3037] T ded9a2cda7b142d79489c8541e8e15f7 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.530397 10416 consensus_queue.cc:237] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.531204 9300 raft_consensus.cc:2454] T ded9a2cda7b142d79489c8541e8e15f7 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 1.
I20240502 14:01:16.531301 9433 raft_consensus.cc:2454] T ded9a2cda7b142d79489c8541e8e15f7 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 1.
I20240502 14:01:16.532442 8889 catalog_manager.cc:5484] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca reported cstate change: term changed from 0 to 1, leader changed from <none> to d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69). New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } }
W20240502 14:01:16.657433 10878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:16.657692 10878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:16.660951 10878 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_5'
I20240502 14:01:16.739444 9169 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.739954 9038 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.740262 10416 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.741068 9829 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.741568 9300 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.741827 9169 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.741966 9697 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.742439 10313 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.742563 10398 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.746562 9170 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.747056 10838 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.748235 9036 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.748631 10829 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.749646 9698 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.750312 10838 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.750507 9829 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.750612 10838 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.750893 10838 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.751214 9300 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.753427 9566 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.753852 9564 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.758953 9038 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.759166 10398 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.759630 10838 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.761068 10838 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.761394 10838 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.776516 10838 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.776863 10838 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.796078 9696 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.796608 10791 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.802588 9302 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.803692 10829 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.812172 9433 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.813792 10825 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:16.820914 9830 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:16.825141 10825 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 199.605 ms
time per row: 0.0998027 ms
I20240502 14:01:16.906929 8895 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46030:
name: "rebalance_test_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240502 14:01:16.907534 8895 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table rebalance_test_table_6 in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20240502 14:01:16.913316 9018 tablet_service.cc:1467] Processing CreateTablet for tablet 0801ed14be134f2a97d33a5c7a8f5cff (DEFAULT_TABLE table=rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.913316 9150 tablet_service.cc:1467] Processing CreateTablet for tablet 0801ed14be134f2a97d33a5c7a8f5cff (DEFAULT_TABLE table=rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.913622 9150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0801ed14be134f2a97d33a5c7a8f5cff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.913661 9018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0801ed14be134f2a97d33a5c7a8f5cff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.913594 9282 tablet_service.cc:1467] Processing CreateTablet for tablet 0801ed14be134f2a97d33a5c7a8f5cff (DEFAULT_TABLE table=rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.913866 9282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0801ed14be134f2a97d33a5c7a8f5cff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.915822 10858 tablet_bootstrap.cc:492] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16: Bootstrap starting.
I20240502 14:01:16.915974 10857 tablet_bootstrap.cc:492] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f: Bootstrap starting.
I20240502 14:01:16.916749 10858 tablet_bootstrap.cc:654] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.916738 9677 tablet_service.cc:1467] Processing CreateTablet for tablet 0801ed14be134f2a97d33a5c7a8f5cff (DEFAULT_TABLE table=rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.916864 10857 tablet_bootstrap.cc:654] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.916886 9546 tablet_service.cc:1467] Processing CreateTablet for tablet 0801ed14be134f2a97d33a5c7a8f5cff (DEFAULT_TABLE table=rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.917023 9677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0801ed14be134f2a97d33a5c7a8f5cff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.917200 9546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0801ed14be134f2a97d33a5c7a8f5cff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.917393 10304 tablet_bootstrap.cc:492] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap starting.
I20240502 14:01:16.918197 10304 tablet_bootstrap.cc:654] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.918859 10760 tablet_bootstrap.cc:492] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap starting.
I20240502 14:01:16.919247 10853 tablet_bootstrap.cc:492] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288: Bootstrap starting.
I20240502 14:01:16.919473 9676 tablet_service.cc:1467] Processing CreateTablet for tablet 6232f47837664d7486612ba1d291f2b8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.919689 10760 tablet_bootstrap.cc:654] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.919708 9676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6232f47837664d7486612ba1d291f2b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.919906 10853 tablet_bootstrap.cc:654] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.920653 9544 tablet_service.cc:1467] Processing CreateTablet for tablet 7289a5be3b3f437f9a2770baa42b89a8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.920872 9544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7289a5be3b3f437f9a2770baa42b89a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.920838 9546 tablet_service.cc:1467] Processing CreateTablet for tablet 6232f47837664d7486612ba1d291f2b8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.920924 10760 tablet_bootstrap.cc:492] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca: No bootstrap required, opened a new log
I20240502 14:01:16.921001 10760 ts_tablet_manager.cc:1367] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:16.921015 9546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6232f47837664d7486612ba1d291f2b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.921428 10760 raft_consensus.cc:348] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:16.921545 10760 raft_consensus.cc:374] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.921587 10760 raft_consensus.cc:729] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.921710 10760 consensus_queue.cc:260] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:16.921808 9678 tablet_service.cc:1467] Processing CreateTablet for tablet 7289a5be3b3f437f9a2770baa42b89a8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.922010 9678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7289a5be3b3f437f9a2770baa42b89a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.922006 10760 ts_tablet_manager.cc:1398] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:16.922750 10760 tablet_bootstrap.cc:492] T 6232f47837664d7486612ba1d291f2b8 P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap starting.
I20240502 14:01:16.923300 10304 tablet_bootstrap.cc:492] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d: No bootstrap required, opened a new log
I20240502 14:01:16.923388 10304 ts_tablet_manager.cc:1367] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20240502 14:01:16.923502 10760 tablet_bootstrap.cc:654] T 6232f47837664d7486612ba1d291f2b8 P d0f45862d7f14ce1976c1a37842f25ca: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.923883 10304 raft_consensus.cc:348] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:16.924008 10304 raft_consensus.cc:374] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.924042 10304 raft_consensus.cc:729] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9e4f2f668d48ea87d5aad4a2fb569d, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.924166 10304 consensus_queue.cc:260] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:16.924512 10304 ts_tablet_manager.cc:1398] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:16.924909 10760 tablet_bootstrap.cc:492] T 6232f47837664d7486612ba1d291f2b8 P d0f45862d7f14ce1976c1a37842f25ca: No bootstrap required, opened a new log
I20240502 14:01:16.924989 10760 ts_tablet_manager.cc:1367] T 6232f47837664d7486612ba1d291f2b8 P d0f45862d7f14ce1976c1a37842f25ca: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:16.925595 9281 tablet_service.cc:1467] Processing CreateTablet for tablet 7289a5be3b3f437f9a2770baa42b89a8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.925866 9281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7289a5be3b3f437f9a2770baa42b89a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.926236 9808 tablet_service.cc:1467] Processing CreateTablet for tablet 0801ed14be134f2a97d33a5c7a8f5cff (DEFAULT_TABLE table=rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.926327 9809 tablet_service.cc:1467] Processing CreateTablet for tablet 6232f47837664d7486612ba1d291f2b8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.926507 9808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0801ed14be134f2a97d33a5c7a8f5cff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.926754 9810 tablet_service.cc:1467] Processing CreateTablet for tablet 7289a5be3b3f437f9a2770baa42b89a8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.926887 10858 tablet_bootstrap.cc:492] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16: No bootstrap required, opened a new log
I20240502 14:01:16.926959 9810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7289a5be3b3f437f9a2770baa42b89a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.926973 10858 ts_tablet_manager.cc:1367] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240502 14:01:16.927430 10858 raft_consensus.cc:348] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:16.927542 10858 raft_consensus.cc:374] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.927577 10858 raft_consensus.cc:729] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83af1bc823a4f2ca531f0d6825f8e16, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.927687 10858 consensus_queue.cc:260] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:16.927999 10858 ts_tablet_manager.cc:1398] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:16.928602 10862 tablet_bootstrap.cc:492] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b: Bootstrap starting.
I20240502 14:01:16.929092 9809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6232f47837664d7486612ba1d291f2b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.929272 10862 tablet_bootstrap.cc:654] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.930562 10760 raft_consensus.cc:348] T 6232f47837664d7486612ba1d291f2b8 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.930676 10760 raft_consensus.cc:374] T 6232f47837664d7486612ba1d291f2b8 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.930709 10760 raft_consensus.cc:729] T 6232f47837664d7486612ba1d291f2b8 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.930819 10760 consensus_queue.cc:260] T 6232f47837664d7486612ba1d291f2b8 P d0f45862d7f14ce1976c1a37842f25ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.931066 10760 ts_tablet_manager.cc:1398] T 6232f47837664d7486612ba1d291f2b8 P d0f45862d7f14ce1976c1a37842f25ca: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240502 14:01:16.931206 10760 tablet_bootstrap.cc:492] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap starting.
I20240502 14:01:16.932334 9018 tablet_service.cc:1467] Processing CreateTablet for tablet 6232f47837664d7486612ba1d291f2b8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.932479 9149 tablet_service.cc:1467] Processing CreateTablet for tablet 7289a5be3b3f437f9a2770baa42b89a8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.932600 9018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6232f47837664d7486612ba1d291f2b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.932698 9149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7289a5be3b3f437f9a2770baa42b89a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.932839 9412 tablet_service.cc:1467] Processing CreateTablet for tablet 0801ed14be134f2a97d33a5c7a8f5cff (DEFAULT_TABLE table=rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.933009 9413 tablet_service.cc:1467] Processing CreateTablet for tablet 7289a5be3b3f437f9a2770baa42b89a8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.933095 9412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0801ed14be134f2a97d33a5c7a8f5cff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.933187 9413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7289a5be3b3f437f9a2770baa42b89a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.932839 9414 tablet_service.cc:1467] Processing CreateTablet for tablet 6232f47837664d7486612ba1d291f2b8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.934365 9414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6232f47837664d7486612ba1d291f2b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.934625 10304 tablet_bootstrap.cc:492] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap starting.
I20240502 14:01:16.935191 10759 tablet_bootstrap.cc:492] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:16.935398 10304 tablet_bootstrap.cc:654] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.935693 9282 tablet_service.cc:1467] Processing CreateTablet for tablet 6232f47837664d7486612ba1d291f2b8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.935920 9282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6232f47837664d7486612ba1d291f2b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.936055 10759 tablet_bootstrap.cc:654] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.936452 9018 tablet_service.cc:1467] Processing CreateTablet for tablet 7289a5be3b3f437f9a2770baa42b89a8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.936651 9018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7289a5be3b3f437f9a2770baa42b89a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.938230 9150 tablet_service.cc:1467] Processing CreateTablet for tablet 6232f47837664d7486612ba1d291f2b8 (DEFAULT_TABLE table=rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 14:01:16.938417 9150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6232f47837664d7486612ba1d291f2b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:16.938931 10862 tablet_bootstrap.cc:492] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b: No bootstrap required, opened a new log
I20240502 14:01:16.939013 10862 ts_tablet_manager.cc:1367] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240502 14:01:16.939119 10857 tablet_bootstrap.cc:492] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f: No bootstrap required, opened a new log
I20240502 14:01:16.939229 10857 ts_tablet_manager.cc:1367] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.000s
I20240502 14:01:16.939309 10858 tablet_bootstrap.cc:492] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16: Bootstrap starting.
I20240502 14:01:16.939481 10862 raft_consensus.cc:348] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:16.939617 10862 raft_consensus.cc:374] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.939648 10862 raft_consensus.cc:729] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.939651 10857 raft_consensus.cc:348] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:16.939776 10857 raft_consensus.cc:374] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.939766 10862 consensus_queue.cc:260] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:16.939810 10857 raft_consensus.cc:729] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.939924 10857 consensus_queue.cc:260] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:16.940039 10862 ts_tablet_manager.cc:1398] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:16.940218 10862 tablet_bootstrap.cc:492] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b: Bootstrap starting.
I20240502 14:01:16.940222 10858 tablet_bootstrap.cc:654] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.940223 10857 ts_tablet_manager.cc:1398] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:16.940441 10857 tablet_bootstrap.cc:492] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f: Bootstrap starting.
I20240502 14:01:16.940984 10862 tablet_bootstrap.cc:654] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.941627 10857 tablet_bootstrap.cc:654] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.942613 10857 tablet_bootstrap.cc:492] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f: No bootstrap required, opened a new log
I20240502 14:01:16.942687 10857 ts_tablet_manager.cc:1367] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:16.943156 10857 raft_consensus.cc:348] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.943279 10857 raft_consensus.cc:374] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.943323 10857 raft_consensus.cc:729] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.943451 10857 consensus_queue.cc:260] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.943768 10857 ts_tablet_manager.cc:1398] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:16.944052 10857 tablet_bootstrap.cc:492] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f: Bootstrap starting.
I20240502 14:01:16.945003 10857 tablet_bootstrap.cc:654] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.945791 10760 tablet_bootstrap.cc:654] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.946816 10304 tablet_bootstrap.cc:492] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d: No bootstrap required, opened a new log
I20240502 14:01:16.946916 10304 ts_tablet_manager.cc:1367] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240502 14:01:16.947094 10858 tablet_bootstrap.cc:492] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16: No bootstrap required, opened a new log
I20240502 14:01:16.947176 10858 ts_tablet_manager.cc:1367] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20240502 14:01:16.947399 10304 raft_consensus.cc:348] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.947530 10304 raft_consensus.cc:374] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.947568 10304 raft_consensus.cc:729] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9e4f2f668d48ea87d5aad4a2fb569d, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.947635 10858 raft_consensus.cc:348] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.947698 10304 consensus_queue.cc:260] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.947763 10858 raft_consensus.cc:374] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.947795 10858 raft_consensus.cc:729] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83af1bc823a4f2ca531f0d6825f8e16, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.947933 10858 consensus_queue.cc:260] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.948001 10304 ts_tablet_manager.cc:1398] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:16.948160 10304 tablet_bootstrap.cc:492] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap starting.
I20240502 14:01:16.948223 10858 ts_tablet_manager.cc:1398] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 14:01:16.948379 10853 tablet_bootstrap.cc:492] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288: No bootstrap required, opened a new log
I20240502 14:01:16.948405 10858 tablet_bootstrap.cc:492] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16: Bootstrap starting.
I20240502 14:01:16.948485 10853 ts_tablet_manager.cc:1367] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.000s
I20240502 14:01:16.948877 10853 raft_consensus.cc:348] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:16.948983 10853 raft_consensus.cc:374] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.949018 10853 raft_consensus.cc:729] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa836edab8d4853a7ca47a23117e288, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.949064 10304 tablet_bootstrap.cc:654] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.949155 10853 consensus_queue.cc:260] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:16.949419 10853 ts_tablet_manager.cc:1398] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:16.949589 10853 tablet_bootstrap.cc:492] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288: Bootstrap starting.
I20240502 14:01:16.950132 10304 tablet_bootstrap.cc:492] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d: No bootstrap required, opened a new log
I20240502 14:01:16.950254 10304 ts_tablet_manager.cc:1367] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:16.950553 10853 tablet_bootstrap.cc:654] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.950850 10304 raft_consensus.cc:348] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.951043 10304 raft_consensus.cc:374] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.951125 10304 raft_consensus.cc:729] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9e4f2f668d48ea87d5aad4a2fb569d, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.951309 10304 consensus_queue.cc:260] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.951442 10857 tablet_bootstrap.cc:492] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f: No bootstrap required, opened a new log
I20240502 14:01:16.951484 10760 tablet_bootstrap.cc:492] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca: No bootstrap required, opened a new log
I20240502 14:01:16.951534 10857 ts_tablet_manager.cc:1367] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 14:01:16.951603 10760 ts_tablet_manager.cc:1367] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20240502 14:01:16.951803 10304 ts_tablet_manager.cc:1398] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:16.952013 10857 raft_consensus.cc:348] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.952054 10760 raft_consensus.cc:348] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.952148 10760 raft_consensus.cc:374] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.952147 10857 raft_consensus.cc:374] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.952179 10760 raft_consensus.cc:729] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.952188 10857 raft_consensus.cc:729] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.952298 10857 consensus_queue.cc:260] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.952322 10760 consensus_queue.cc:260] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.952626 10857 ts_tablet_manager.cc:1398] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:16.952771 10760 ts_tablet_manager.cc:1398] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:16.953122 10862 tablet_bootstrap.cc:492] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b: No bootstrap required, opened a new log
I20240502 14:01:16.953204 10862 ts_tablet_manager.cc:1367] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 14:01:16.953665 10862 raft_consensus.cc:348] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.953828 10862 raft_consensus.cc:374] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.953864 10862 raft_consensus.cc:729] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.953989 10862 consensus_queue.cc:260] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.954277 10862 ts_tablet_manager.cc:1398] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:16.954433 10862 tablet_bootstrap.cc:492] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b: Bootstrap starting.
I20240502 14:01:16.954473 10858 tablet_bootstrap.cc:654] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.955322 10862 tablet_bootstrap.cc:654] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.955749 10858 tablet_bootstrap.cc:492] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16: No bootstrap required, opened a new log
I20240502 14:01:16.955832 10858 ts_tablet_manager.cc:1367] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20240502 14:01:16.956723 10853 tablet_bootstrap.cc:492] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288: No bootstrap required, opened a new log
I20240502 14:01:16.956820 10853 ts_tablet_manager.cc:1367] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 14:01:16.957114 10759 tablet_bootstrap.cc:492] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de: No bootstrap required, opened a new log
I20240502 14:01:16.957208 10759 ts_tablet_manager.cc:1367] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.000s
I20240502 14:01:16.957305 10853 raft_consensus.cc:348] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.957424 10853 raft_consensus.cc:374] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.957451 10853 raft_consensus.cc:729] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa836edab8d4853a7ca47a23117e288, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.957576 10853 consensus_queue.cc:260] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.957680 10759 raft_consensus.cc:348] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.957809 10759 raft_consensus.cc:374] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.957850 10759 raft_consensus.cc:729] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.957867 10853 ts_tablet_manager.cc:1398] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:16.957804 10858 raft_consensus.cc:348] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.957960 10858 raft_consensus.cc:374] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.958000 10858 raft_consensus.cc:729] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83af1bc823a4f2ca531f0d6825f8e16, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.957973 10759 consensus_queue.cc:260] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.958042 10853 tablet_bootstrap.cc:492] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288: Bootstrap starting.
I20240502 14:01:16.958119 10858 consensus_queue.cc:260] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.958251 10759 ts_tablet_manager.cc:1398] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:16.958413 10858 ts_tablet_manager.cc:1398] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 14:01:16.958433 10759 tablet_bootstrap.cc:492] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:16.958897 10853 tablet_bootstrap.cc:654] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.959127 10862 tablet_bootstrap.cc:492] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b: No bootstrap required, opened a new log
I20240502 14:01:16.959205 10862 ts_tablet_manager.cc:1367] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 14:01:16.959287 10759 tablet_bootstrap.cc:654] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.959899 10862 raft_consensus.cc:348] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.960044 10862 raft_consensus.cc:374] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.960083 10862 raft_consensus.cc:729] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.960471 10759 tablet_bootstrap.cc:492] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de: No bootstrap required, opened a new log
I20240502 14:01:16.960218 10862 consensus_queue.cc:260] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.960595 10759 ts_tablet_manager.cc:1367] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:16.960819 10862 ts_tablet_manager.cc:1398] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:16.961314 10759 raft_consensus.cc:348] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.961505 10759 raft_consensus.cc:374] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.961601 10759 raft_consensus.cc:729] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.961807 10759 consensus_queue.cc:260] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.962083 10853 tablet_bootstrap.cc:492] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288: No bootstrap required, opened a new log
I20240502 14:01:16.962172 10853 ts_tablet_manager.cc:1367] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240502 14:01:16.962337 10759 ts_tablet_manager.cc:1398] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:01:16.962533 10759 tablet_bootstrap.cc:492] T 0801ed14be134f2a97d33a5c7a8f5cff P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:16.962605 10853 raft_consensus.cc:348] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.962728 10853 raft_consensus.cc:374] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.962757 10853 raft_consensus.cc:729] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa836edab8d4853a7ca47a23117e288, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.962862 10853 consensus_queue.cc:260] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.963136 10853 ts_tablet_manager.cc:1398] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:16.963465 10759 tablet_bootstrap.cc:654] T 0801ed14be134f2a97d33a5c7a8f5cff P 1a490f170c614ee4900ef16c3287c7de: Neither blocks nor log segments found. Creating new log.
I20240502 14:01:16.964476 10759 tablet_bootstrap.cc:492] T 0801ed14be134f2a97d33a5c7a8f5cff P 1a490f170c614ee4900ef16c3287c7de: No bootstrap required, opened a new log
I20240502 14:01:16.964577 10759 ts_tablet_manager.cc:1367] T 0801ed14be134f2a97d33a5c7a8f5cff P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 14:01:16.965014 10759 raft_consensus.cc:348] T 0801ed14be134f2a97d33a5c7a8f5cff P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:16.965118 10759 raft_consensus.cc:374] T 0801ed14be134f2a97d33a5c7a8f5cff P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 14:01:16.965148 10759 raft_consensus.cc:729] T 0801ed14be134f2a97d33a5c7a8f5cff P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: FOLLOWER
I20240502 14:01:16.965272 10759 consensus_queue.cc:260] T 0801ed14be134f2a97d33a5c7a8f5cff P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:16.965553 10759 ts_tablet_manager.cc:1398] T 0801ed14be134f2a97d33a5c7a8f5cff P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:16.970255 10546 raft_consensus.cc:482] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:16.970364 10546 raft_consensus.cc:504] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:16.971011 10546 leader_election.cc:288] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:16.971119 9433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0801ed14be134f2a97d33a5c7a8f5cff" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de" is_pre_election: true
I20240502 14:01:16.971114 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0801ed14be134f2a97d33a5c7a8f5cff" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" is_pre_election: true
I20240502 14:01:16.971218 9433 raft_consensus.cc:2454] T 0801ed14be134f2a97d33a5c7a8f5cff P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 0.
I20240502 14:01:16.971256 9038 raft_consensus.cc:2454] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 0.
I20240502 14:01:16.971315 9169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0801ed14be134f2a97d33a5c7a8f5cff" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" is_pre_election: true
I20240502 14:01:16.971426 9169 raft_consensus.cc:2454] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 0.
I20240502 14:01:16.971854 9696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0801ed14be134f2a97d33a5c7a8f5cff" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288" is_pre_election: true
I20240502 14:01:16.971974 9696 raft_consensus.cc:2454] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 0.
I20240502 14:01:16.972018 9237 leader_election.cc:302] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1a490f170c614ee4900ef16c3287c7de, bc9e4f2f668d48ea87d5aad4a2fb569d, ccfb5bbc1bb54793812d11170ac2d25f, d83af1bc823a4f2ca531f0d6825f8e16; no voters:
I20240502 14:01:16.972036 9566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0801ed14be134f2a97d33a5c7a8f5cff" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca" is_pre_election: true
I20240502 14:01:16.972189 10546 raft_consensus.cc:2781] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:16.972187 9566 raft_consensus.cc:2454] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 0.
I20240502 14:01:16.972236 10546 raft_consensus.cc:482] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:16.972277 10546 raft_consensus.cc:3037] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.972889 9829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0801ed14be134f2a97d33a5c7a8f5cff" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b" is_pre_election: true
I20240502 14:01:16.973014 9829 raft_consensus.cc:2454] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 0.
I20240502 14:01:16.973297 10546 raft_consensus.cc:504] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:16.973788 10546 leader_election.cc:288] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [CANDIDATE]: Term 1 election: Requested vote from peers bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:16.973994 9169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0801ed14be134f2a97d33a5c7a8f5cff" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16"
I20240502 14:01:16.974005 9433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0801ed14be134f2a97d33a5c7a8f5cff" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de"
I20240502 14:01:16.974093 9169 raft_consensus.cc:3037] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.974050 9566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0801ed14be134f2a97d33a5c7a8f5cff" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca"
I20240502 14:01:16.974107 9433 raft_consensus.cc:3037] T 0801ed14be134f2a97d33a5c7a8f5cff P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.974172 9566 raft_consensus.cc:3037] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.974174 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0801ed14be134f2a97d33a5c7a8f5cff" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d"
I20240502 14:01:16.974270 9038 raft_consensus.cc:3037] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.975028 9169 raft_consensus.cc:2454] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 1.
I20240502 14:01:16.975057 9696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0801ed14be134f2a97d33a5c7a8f5cff" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288"
I20240502 14:01:16.975116 9433 raft_consensus.cc:2454] T 0801ed14be134f2a97d33a5c7a8f5cff P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 1.
I20240502 14:01:16.975157 9696 raft_consensus.cc:3037] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.975169 9566 raft_consensus.cc:2454] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 1.
I20240502 14:01:16.975960 9696 raft_consensus.cc:2454] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 1.
I20240502 14:01:16.976284 10825 raft_consensus.cc:482] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:16.976361 9236 leader_election.cc:302] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 1a490f170c614ee4900ef16c3287c7de, ccfb5bbc1bb54793812d11170ac2d25f, d0f45862d7f14ce1976c1a37842f25ca, d83af1bc823a4f2ca531f0d6825f8e16; no voters:
I20240502 14:01:16.976442 10825 raft_consensus.cc:504] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.976632 10546 raft_consensus.cc:2781] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:16.976796 10546 raft_consensus.cc:686] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Becoming Leader. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Running, Role: LEADER
I20240502 14:01:16.977061 9038 raft_consensus.cc:2454] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 1.
I20240502 14:01:16.977169 10825 leader_election.cc:288] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:16.977602 9830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0801ed14be134f2a97d33a5c7a8f5cff" candidate_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b"
I20240502 14:01:16.977687 9829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7289a5be3b3f437f9a2770baa42b89a8" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b" is_pre_election: true
I20240502 14:01:16.977718 9830 raft_consensus.cc:3037] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.977713 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7289a5be3b3f437f9a2770baa42b89a8" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" is_pre_election: true
I20240502 14:01:16.977772 9829 raft_consensus.cc:2454] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 0.
I20240502 14:01:16.977825 9038 raft_consensus.cc:2454] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 0.
I20240502 14:01:16.977880 9302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7289a5be3b3f437f9a2770baa42b89a8" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" is_pre_election: true
I20240502 14:01:16.977980 9302 raft_consensus.cc:2454] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 0.
I20240502 14:01:16.978194 9499 leader_election.cc:302] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0488b0e4ed994db19d40aed8c15f945b, bc9e4f2f668d48ea87d5aad4a2fb569d, ccfb5bbc1bb54793812d11170ac2d25f, d0f45862d7f14ce1976c1a37842f25ca; no voters:
I20240502 14:01:16.978358 10825 raft_consensus.cc:2781] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:16.978399 10825 raft_consensus.cc:482] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:16.978431 10825 raft_consensus.cc:3037] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.978749 9830 raft_consensus.cc:2454] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccfb5bbc1bb54793812d11170ac2d25f in term 1.
I20240502 14:01:16.979373 10825 raft_consensus.cc:504] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.979888 10825 leader_election.cc:288] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [CANDIDATE]: Term 1 election: Requested vote from peers 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:16.980118 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7289a5be3b3f437f9a2770baa42b89a8" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d"
I20240502 14:01:16.980103 9168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7289a5be3b3f437f9a2770baa42b89a8" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16"
I20240502 14:01:16.980175 9302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7289a5be3b3f437f9a2770baa42b89a8" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f"
I20240502 14:01:16.980175 9697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7289a5be3b3f437f9a2770baa42b89a8" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288"
I20240502 14:01:16.980211 9038 raft_consensus.cc:3037] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.980211 9168 raft_consensus.cc:3037] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.980270 9697 raft_consensus.cc:3037] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.980270 9302 raft_consensus.cc:3037] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.981091 9038 raft_consensus.cc:2454] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 1.
I20240502 14:01:16.981144 9168 raft_consensus.cc:2454] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 1.
I20240502 14:01:16.981146 9302 raft_consensus.cc:2454] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 1.
I20240502 14:01:16.981148 9697 raft_consensus.cc:2454] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 1.
I20240502 14:01:16.981508 9499 leader_election.cc:302] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: bc9e4f2f668d48ea87d5aad4a2fb569d, ccfb5bbc1bb54793812d11170ac2d25f, d0f45862d7f14ce1976c1a37842f25ca, d83af1bc823a4f2ca531f0d6825f8e16; no voters:
I20240502 14:01:16.981536 9830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7289a5be3b3f437f9a2770baa42b89a8" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0488b0e4ed994db19d40aed8c15f945b"
I20240502 14:01:16.981648 9830 raft_consensus.cc:3037] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.981907 9433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7289a5be3b3f437f9a2770baa42b89a8" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de" is_pre_election: true
I20240502 14:01:16.982033 9433 raft_consensus.cc:2454] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 0.
I20240502 14:01:16.982122 9696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7289a5be3b3f437f9a2770baa42b89a8" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288" is_pre_election: true
I20240502 14:01:16.982250 9696 raft_consensus.cc:2362] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 1. Re-sending same reply.
I20240502 14:01:16.982017 9434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7289a5be3b3f437f9a2770baa42b89a8" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de"
I20240502 14:01:16.982508 9434 raft_consensus.cc:3037] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.982530 9830 raft_consensus.cc:2454] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 1.
I20240502 14:01:16.983336 9434 raft_consensus.cc:2454] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 1.
I20240502 14:01:16.983777 9169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7289a5be3b3f437f9a2770baa42b89a8" candidate_uuid: "d0f45862d7f14ce1976c1a37842f25ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" is_pre_election: true
I20240502 14:01:16.983902 9169 raft_consensus.cc:2362] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d0f45862d7f14ce1976c1a37842f25ca in term 1. Re-sending same reply.
I20240502 14:01:16.984077 10825 raft_consensus.cc:2781] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:16.984177 10825 raft_consensus.cc:686] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Becoming Leader. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Running, Role: LEADER
I20240502 14:01:16.984369 10825 consensus_queue.cc:237] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.976930 10546 consensus_queue.cc:237] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:16.985941 10718 raft_consensus.cc:482] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 14:01:16.986023 10718 raft_consensus.cc:504] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.986711 10718 leader_election.cc:288] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:16.986801 8895 catalog_manager.cc:5484] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca reported cstate change: term changed from 0 to 1, leader changed from <none> to d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69). New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.987002 9302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6232f47837664d7486612ba1d291f2b8" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" is_pre_election: true
I20240502 14:01:16.987100 9302 raft_consensus.cc:2454] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:16.987100 9169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6232f47837664d7486612ba1d291f2b8" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" is_pre_election: true
I20240502 14:01:16.987215 9169 raft_consensus.cc:2454] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:16.987603 9434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6232f47837664d7486612ba1d291f2b8" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de" is_pre_election: true
I20240502 14:01:16.987617 9696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6232f47837664d7486612ba1d291f2b8" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288" is_pre_election: true
I20240502 14:01:16.987725 9434 raft_consensus.cc:2454] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:16.987730 9696 raft_consensus.cc:2454] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:16.987985 9765 leader_election.cc:302] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0488b0e4ed994db19d40aed8c15f945b, 1a490f170c614ee4900ef16c3287c7de, ccfb5bbc1bb54793812d11170ac2d25f, d83af1bc823a4f2ca531f0d6825f8e16; no voters:
I20240502 14:01:16.988067 9566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6232f47837664d7486612ba1d291f2b8" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca" is_pre_election: true
I20240502 14:01:16.988168 10718 raft_consensus.cc:2781] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 14:01:16.988170 9566 raft_consensus.cc:2454] T 6232f47837664d7486612ba1d291f2b8 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:16.988212 10718 raft_consensus.cc:482] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 14:01:16.988246 10718 raft_consensus.cc:3037] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.989038 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6232f47837664d7486612ba1d291f2b8" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" is_pre_election: true
I20240502 14:01:16.989166 9038 raft_consensus.cc:2454] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 0.
I20240502 14:01:16.989301 10718 raft_consensus.cc:504] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.989593 8889 catalog_manager.cc:5484] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: term changed from 0 to 1, leader changed from <none> to ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67). New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:16.989909 10718 leader_election.cc:288] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 election: Requested vote from peers efa836edab8d4853a7ca47a23117e288 (127.24.66.70:43397), d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375), 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907), bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459), d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837), ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:16.990018 9566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6232f47837664d7486612ba1d291f2b8" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f45862d7f14ce1976c1a37842f25ca"
I20240502 14:01:16.990137 9566 raft_consensus.cc:3037] T 6232f47837664d7486612ba1d291f2b8 P d0f45862d7f14ce1976c1a37842f25ca [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.990190 9302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6232f47837664d7486612ba1d291f2b8" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccfb5bbc1bb54793812d11170ac2d25f"
I20240502 14:01:16.990291 9302 raft_consensus.cc:3037] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.991120 9566 raft_consensus.cc:2454] T 6232f47837664d7486612ba1d291f2b8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:16.991243 9302 raft_consensus.cc:2454] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:16.991271 9169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6232f47837664d7486612ba1d291f2b8" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d83af1bc823a4f2ca531f0d6825f8e16"
I20240502 14:01:16.991385 9169 raft_consensus.cc:3037] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.991503 9434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6232f47837664d7486612ba1d291f2b8" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a490f170c614ee4900ef16c3287c7de"
I20240502 14:01:16.991632 9434 raft_consensus.cc:3037] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.992058 9696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6232f47837664d7486612ba1d291f2b8" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa836edab8d4853a7ca47a23117e288"
I20240502 14:01:16.992173 9696 raft_consensus.cc:3037] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288 [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.992453 9169 raft_consensus.cc:2454] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:16.992655 9038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6232f47837664d7486612ba1d291f2b8" candidate_uuid: "0488b0e4ed994db19d40aed8c15f945b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d"
I20240502 14:01:16.992734 9765 leader_election.cc:302] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 0488b0e4ed994db19d40aed8c15f945b, ccfb5bbc1bb54793812d11170ac2d25f, d0f45862d7f14ce1976c1a37842f25ca, d83af1bc823a4f2ca531f0d6825f8e16; no voters:
I20240502 14:01:16.992770 9038 raft_consensus.cc:3037] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 0 FOLLOWER]: Advancing to term 1
I20240502 14:01:16.992892 10718 raft_consensus.cc:2781] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 14:01:16.993198 9696 raft_consensus.cc:2454] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:16.993479 10718 raft_consensus.cc:686] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Becoming Leader. State: Replica: 0488b0e4ed994db19d40aed8c15f945b, State: Running, Role: LEADER
I20240502 14:01:16.993639 10718 consensus_queue.cc:237] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:16.994096 9038 raft_consensus.cc:2454] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:16.994386 9434 raft_consensus.cc:2454] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0488b0e4ed994db19d40aed8c15f945b in term 1.
I20240502 14:01:16.995833 8889 catalog_manager.cc:5484] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71). New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } }
W20240502 14:01:17.131317 10956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:17.131549 10956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:17.134580 10956 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_6'
I20240502 14:01:17.218153 9829 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:17.218580 10546 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:17.219671 9697 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:17.219769 9302 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:17.220059 9696 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:17.220247 9566 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:17.220443 9434 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:17.221710 10708 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:17.221765 9169 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:17.221997 10708 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:17.222146 9038 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:17.222618 10712 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:17.222831 9038 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:17.222934 10712 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:17.226022 9695 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:17.226394 9830 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:17.226612 10416 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:17.227856 9169 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:17.228536 9433 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:17.229586 9169 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:17.230564 10829 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:17.231381 10546 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:17.231709 10546 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:17.231843 10712 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:17.232621 9302 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:17.233186 9038 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:17.233395 9565 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 14:01:17.234100 10416 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:17.235867 9433 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 14:01:17.236071 10829 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:17.242240 10718 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:17.242547 10718 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:17.242784 10718 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:17.244486 10829 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:17.245018 10829 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:17.251359 10993 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 14:01:17.350658 10421 mvcc.cc:205] Tried to move back new op lower bound from 7023241122846789632 to 7023241121731100672. Current Snapshot: MvccSnapshot[applied={T|T < 7023241122846789632}]
INSERT report
rows total: 2000
time total: 189.418 ms
time per row: 0.0947089 ms
I20240502 14:01:17.380165 10428 mvcc.cc:205] Tried to move back new op lower bound from 7023241122683236352 to 7023241121769201664. Current Snapshot: MvccSnapshot[applied={T|T < 7023241122683236352}]
I20240502 14:01:17.400776 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.73:42559
--local_ip_for_outbound_sockets=127.24.66.73
--tserver_master_addrs=127.24.66.126:40003
--webserver_port=35739
--webserver_interface=127.24.66.73
--builtin_ntp_servers=127.24.66.84:34375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20240502 14:01:17.530550 11018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:17.530759 11018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:17.530803 11018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:17.534042 11018 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:17.534106 11018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:17.534205 11018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.73
I20240502 14:01:17.537707 11018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.73:42559
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/data/info.pb
--webserver_interface=127.24.66.73
--webserver_port=35739
--tserver_master_addrs=127.24.66.126:40003
--never_fsync=true
--heap_profile_path=/tmp/kudu.11018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.73
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:17.538580 11018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:17.539857 11018 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:17.548267 11024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:17.548846 11023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:17.549592 11026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:17.550254 11018 server_base.cc:1034] running on GCE node
I20240502 14:01:17.550640 11018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:17.551163 11018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:17.552309 11018 hybrid_clock.cc:648] HybridClock initialized: now 1714658477552287 us; error 38 us; skew 500 ppm
I20240502 14:01:17.554291 11018 webserver.cc:460] Webserver started at http://127.24.66.73:35739/ using document root <none> and password file <none>
I20240502 14:01:17.554951 11018 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:17.555037 11018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:17.558740 11018 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20240502 14:01:17.560765 11032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:17.561833 11018 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:17.561928 11018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/wal
uuid: "304274b88d234198a3892e38c05014a5"
format_stamp: "Formatted at 2024-05-02 14:01:09 on dist-test-slave-5npg"
I20240502 14:01:17.562346 11018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:17.581197 11018 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:17.581776 11018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:17.581926 11018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:17.582458 11018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:17.583357 11018 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:17.583403 11018 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:17.583448 11018 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:17.583478 11018 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:17.591830 11018 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.73:42559
I20240502 14:01:17.591852 11145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.73:42559 every 8 connection(s)
I20240502 14:01:17.592909 11018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/data/info.pb
I20240502 14:01:17.597867 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 11018
I20240502 14:01:17.598325 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.72:35005
--local_ip_for_outbound_sockets=127.24.66.72
--tserver_master_addrs=127.24.66.126:40003
--webserver_port=36129
--webserver_interface=127.24.66.72
--builtin_ntp_servers=127.24.66.84:34375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:01:17.601945 11146 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40003
I20240502 14:01:17.602207 11146 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:17.602807 11146 heartbeater.cc:511] Master 127.24.66.126:40003 requested a full tablet report, sending...
I20240502 14:01:17.604164 8896 ts_manager.cc:194] Re-registered known tserver with Master: 304274b88d234198a3892e38c05014a5 (127.24.66.73:42559)
I20240502 14:01:17.604920 8896 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.73:36815
W20240502 14:01:17.712352 11149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:17.712564 11149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:17.712625 11149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:17.715776 11149 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:17.715842 11149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:17.715942 11149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.72
I20240502 14:01:17.720064 11149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.72:35005
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.24.66.72
--webserver_port=36129
--tserver_master_addrs=127.24.66.126:40003
--never_fsync=true
--heap_profile_path=/tmp/kudu.11149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.72
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:17.721407 11149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:17.722676 11149 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:17.731256 11157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:17.731844 11154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:17.732504 11155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:17.733214 11149 server_base.cc:1034] running on GCE node
I20240502 14:01:17.733635 11149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:17.734169 11149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:17.735324 11149 hybrid_clock.cc:648] HybridClock initialized: now 1714658477735303 us; error 43 us; skew 500 ppm
I20240502 14:01:17.737464 11149 webserver.cc:460] Webserver started at http://127.24.66.72:36129/ using document root <none> and password file <none>
I20240502 14:01:17.738122 11149 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:17.738204 11149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:17.741711 11149 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 14:01:17.743932 11163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:17.745065 11149 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:17.745173 11149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "e01cf117ef5d446590c06dd1a6ab7ef7"
format_stamp: "Formatted at 2024-05-02 14:01:09 on dist-test-slave-5npg"
I20240502 14:01:17.745674 11149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:17.764487 11149 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:17.765172 11149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:17.765334 11149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:17.765925 11149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:17.767037 11149 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:17.767086 11149 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:17.767128 11149 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:17.767158 11149 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:17.775895 11149 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.72:35005
I20240502 14:01:17.775923 11276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.72:35005 every 8 connection(s)
I20240502 14:01:17.777014 11149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 14:01:17.785919 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 11149
I20240502 14:01:17.786597 24841 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
/tmp/dist-test-taskfX8GOh/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/wal
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/logs
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.66.74:34421
--local_ip_for_outbound_sockets=127.24.66.74
--tserver_master_addrs=127.24.66.126:40003
--webserver_port=40941
--webserver_interface=127.24.66.74
--builtin_ntp_servers=127.24.66.84:34375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20240502 14:01:17.788843 11277 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40003
I20240502 14:01:17.789095 11277 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:17.789785 11277 heartbeater.cc:511] Master 127.24.66.126:40003 requested a full tablet report, sending...
I20240502 14:01:17.790968 8896 ts_manager.cc:194] Re-registered known tserver with Master: e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72:35005)
I20240502 14:01:17.791746 8896 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.72:60309
W20240502 14:01:17.918043 11280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 14:01:17.918370 11280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 14:01:17.918453 11280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 14:01:17.921696 11280 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20240502 14:01:17.921767 11280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 14:01:17.921857 11280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.66.74
I20240502 14:01:17.925761 11280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.66.84:34375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/data
--fs_wal_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.66.74:34421
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/data/info.pb
--webserver_interface=127.24.66.74
--webserver_port=40941
--tserver_master_addrs=127.24.66.126:40003
--never_fsync=true
--heap_profile_path=/tmp/kudu.11280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.66.74
--log_dir=/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision a6923a82b532b3d5c86547dfd9b0772bc97ae884
build type DEBUG
built by None at 02 May 2024 13:43:11 UTC on 1a15035a6830
build id 89
I20240502 14:01:17.926625 11280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 14:01:17.927839 11280 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 14:01:17.936406 11288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:17.936980 11285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 14:01:17.937647 11286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 14:01:17.938292 11280 server_base.cc:1034] running on GCE node
I20240502 14:01:17.938647 11280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 14:01:17.939144 11280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 14:01:17.940295 11280 hybrid_clock.cc:648] HybridClock initialized: now 1714658477940275 us; error 27 us; skew 500 ppm
I20240502 14:01:17.942174 11280 webserver.cc:460] Webserver started at http://127.24.66.74:40941/ using document root <none> and password file <none>
I20240502 14:01:17.942698 11280 fs_manager.cc:359] Metadata directory not provided
I20240502 14:01:17.942763 11280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 14:01:17.946398 11280 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:17.948217 11294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:17.949268 11280 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 14:01:17.949366 11280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/data,/tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/wal
uuid: "535b19a469c34efaaf49ca1cc2e1e5fc"
format_stamp: "Formatted at 2024-05-02 14:01:09 on dist-test-slave-5npg"
I20240502 14:01:17.949787 11280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/wal
metadata directory: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/wal
1 data directories: /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 14:01:17.978626 11280 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 14:01:17.979270 11280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 14:01:17.979429 11280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 14:01:17.979980 11280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 14:01:17.980955 11280 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 14:01:17.981004 11280 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:17.981053 11280 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 14:01:17.981083 11280 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 14:01:17.989193 11280 rpc_server.cc:307] RPC server started. Bound to: 127.24.66.74:34421
I20240502 14:01:17.989228 11407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.66.74:34421 every 8 connection(s)
I20240502 14:01:17.990192 11280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/data/info.pb
I20240502 14:01:17.995962 24841 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfX8GOh/build/debug/bin/kudu as pid 11280
I20240502 14:01:17.998842 11408 heartbeater.cc:347] Connected to a master server at 127.24.66.126:40003
I20240502 14:01:17.999058 11408 heartbeater.cc:464] Registering TS with master...
I20240502 14:01:17.999639 11408 heartbeater.cc:511] Master 127.24.66.126:40003 requested a full tablet report, sending...
I20240502 14:01:18.000661 8896 ts_manager.cc:194] Re-registered known tserver with Master: 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74:34421)
I20240502 14:01:18.001466 8896 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.66.74:40133
I20240502 14:01:18.235371 9412 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:01:18.243427 9544 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:01:18.243940 9018 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:01:18.245114 9809 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:01:18.246210 11080 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:18.258903 9678 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:18.265659 11342 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:18.268877 11211 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:18.274608 9282 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:01:18.305342 9150 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:01:18.471203 11342 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:18.473836 11080 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:18.478706 9150 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:01:18.487301 11211 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:18.490007 9282 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:01:18.495786 9544 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:01:18.496639 9678 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:18.505383 9412 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:01:18.525652 9809 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:01:18.526584 9018 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:01:18.606844 11146 heartbeater.cc:503] Master 127.24.66.126:40003 was elected leader, sending a full tablet report...
I20240502 14:01:18.664471 9809 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:01:18.668285 11080 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:18.670046 9412 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:01:18.670326 9018 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:01:18.680115 11211 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:18.684376 9282 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:01:18.716828 11342 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:18.716967 9544 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:01:18.721038 9150 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:01:18.727222 9678 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:18.773957 9431 consensus_queue.cc:237] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:18.775789 9696 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.776047 9564 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.776222 11560 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.776553 9036 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.776978 11561 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.778272 9168 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.778702 11559 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.779307 11559 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.780578 9828 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.780977 9301 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.781363 11440 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.781538 11560 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.782280 11560 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:18.783274 9168 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:18.785462 9828 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:18.786950 9036 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:18.787205 9564 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:18.787647 8888 catalog_manager.cc:5484] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:18.791923 9696 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:18.794565 11277 heartbeater.cc:503] Master 127.24.66.126:40003 was elected leader, sending a full tablet report...
I20240502 14:01:18.795421 9301 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:18.799593 9301 consensus_queue.cc:237] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } }
I20240502 14:01:18.799835 9433 consensus_queue.cc:237] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:18.802011 9828 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.802400 9696 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.802594 10546 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.802665 9036 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.802951 9828 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.803025 9168 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.803064 9564 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.803246 11559 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.803472 10546 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.803505 11560 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.803642 11559 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.803864 9301 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.803951 9564 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.805320 9696 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.805655 9168 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.806010 11568 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.806463 10546 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.806797 10546 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.806993 9433 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.807165 11568 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.807204 9036 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.807813 11560 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.807822 11440 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:18.808153 9829 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:18.808645 11464 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.810551 9696 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:18.811968 9564 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:18.812811 11583 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.813617 8882 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 9eef89d7e129492bab305a1b5009aace with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 14:01:18.813724 9168 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:18.815177 9036 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:18.815994 8893 catalog_manager.cc:5484] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca reported cstate change: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:18.817978 9301 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:18.821229 11582 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } }
I20240502 14:01:18.821699 9697 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } }
I20240502 14:01:18.823340 9564 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } }
I20240502 14:01:18.825356 9038 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } }
I20240502 14:01:18.826156 8889 catalog_manager.cc:5484] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } }
W20240502 14:01:18.827914 9369 consensus_peers.cc:485] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de -> Peer 304274b88d234198a3892e38c05014a5 (127.24.66.73:42559): Couldn't send request to peer 304274b88d234198a3892e38c05014a5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9eef89d7e129492bab305a1b5009aace. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:18.828985 9433 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } }
I20240502 14:01:18.829066 9170 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } }
I20240502 14:01:18.834614 9828 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } }
I20240502 14:01:18.835354 9301 consensus_queue.cc:237] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:18.837577 9564 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.837622 9433 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.837881 9170 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.838196 10546 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.838471 10546 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.838506 9036 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.838500 9697 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.839005 10546 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.839486 9828 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.839699 11582 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.839864 11464 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.839946 11582 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.842727 10546 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:18.844038 9697 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:18.844736 8880 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet bfa2e6af0fd544fa8bfdaefedabb5e68 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 14:01:18.845139 9828 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:18.846246 9036 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:18.846889 9564 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:18.848570 9170 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:18.849836 8889 catalog_manager.cc:5484] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:18.851223 9433 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
W20240502 14:01:18.860023 9237 consensus_peers.cc:485] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f -> Peer 304274b88d234198a3892e38c05014a5 (127.24.66.73:42559): Couldn't send request to peer 304274b88d234198a3892e38c05014a5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bfa2e6af0fd544fa8bfdaefedabb5e68. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:18.865258 9828 consensus_queue.cc:237] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:18.867120 9433 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.867458 9170 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.867702 9301 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.867765 9697 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.867918 10990 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.867976 9036 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.868108 10989 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.868357 11600 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.868526 10989 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.868839 10989 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.869124 9564 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.869599 10989 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.872363 11602 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:18.872918 9170 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:18.873512 9566 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:18.873512 9038 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:18.875061 9697 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:18.875442 9433 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:18.877475 8889 catalog_manager.cc:5484] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:18.878224 9301 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:18.884238 9828 consensus_queue.cc:237] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:18.886884 9301 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.886957 9697 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.887353 11600 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.887553 10990 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.888000 9433 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.888312 9038 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.888654 9170 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.888813 11600 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.889566 10990 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.890102 9566 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.890938 11614 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.891615 11615 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.891945 11615 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:18.894548 9301 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:18.896242 9696 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:18.898162 8880 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
W20240502 14:01:18.898195 9765 consensus_peers.cc:485] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b -> Peer 304274b88d234198a3892e38c05014a5 (127.24.66.73:42559): Couldn't send request to peer 304274b88d234198a3892e38c05014a5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 97fa16722a1f4adcbf8a5dc3fdfab77a. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:18.898536 8895 catalog_manager.cc:5484] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:18.899940 9433 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:18.899952 9170 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:18.902771 9038 consensus_queue.cc:237] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:18.903915 9566 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:18.905050 9170 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.905447 10618 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.906667 9566 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.906697 9697 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.907114 10618 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.907166 9828 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.907500 10618 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.907838 10618 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.908394 9301 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.909091 9038 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:18.911746 9433 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.911696 10618 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.912297 11620 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.917142 11618 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:18.917719 9697 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:18.917719 9301 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:18.917708 9170 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:18.921195 9433 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:18.921801 8896 catalog_manager.cc:5484] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:18.923151 9564 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:18.927188 9828 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:18.930893 9038 consensus_queue.cc:237] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:18.933709 9697 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.934110 9301 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.934592 9564 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.934598 9170 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.934628 10618 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.935106 9433 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.935508 10618 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.935890 11618 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.936067 9828 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.936793 11618 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.937175 11618 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.937394 11634 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.939414 10618 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:18.940694 9170 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:18.940934 9433 consensus_queue.cc:237] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:18.943053 9038 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.943220 8882 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet f904eef49f9143cd957c8fa5c8ea25ea with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 14:01:18.943691 11585 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.944315 9170 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.944272 8895 catalog_manager.cc:5484] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New cstate: current_term: 1 leader_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:18.945487 9301 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:18.945818 9300 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.946419 11561 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 14:01:18.947408 8974 consensus_peers.cc:485] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d -> Peer 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74:34421): Couldn't send request to peer 535b19a469c34efaaf49ca1cc2e1e5fc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f904eef49f9143cd957c8fa5c8ea25ea. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:18.948246 9828 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.948578 9566 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.948961 11584 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.948912 9564 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:18.950397 9696 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.950789 9830 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:18.951655 11585 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.952988 9697 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:18.954391 11585 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.954855 11585 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:18.955852 9433 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:18.957643 11561 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:18.958292 9697 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:18.959825 9300 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:18.960095 9170 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:18.961652 8889 catalog_manager.cc:5484] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:18.963311 9038 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:18.968081 9564 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:18.968302 9828 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:18.969043 9433 consensus_queue.cc:237] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:18.971951 9828 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.972373 11585 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.973160 9300 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.971896 9697 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.973923 11559 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.974251 9038 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.974368 9564 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.974421 9170 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:18.974925 11585 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.975278 11561 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.976778 11585 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.977118 11585 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:18.978237 11561 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:18.979913 9170 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
W20240502 14:01:18.980223 9370 consensus_peers.cc:485] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de -> Peer e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72:35005): Couldn't send request to peer e01cf117ef5d446590c06dd1a6ab7ef7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8c10fb9748c747c6a89066fa113cda3c. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:18.980556 9696 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:18.981109 9038 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:18.982618 8882 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 8c10fb9748c747c6a89066fa113cda3c with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 14:01:18.984273 8890 catalog_manager.cc:5484] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:18.984483 9828 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:18.984980 9564 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:18.987460 9300 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:18.997581 9170 consensus_queue.cc:237] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:18.999724 9433 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:18.999881 9828 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:19.000053 9300 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:19.000265 11649 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:19.000629 10637 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:19.000859 9038 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:19.000959 10637 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:19.001034 9696 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:19.001298 10637 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:19.001632 10637 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:19.002579 9564 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:19.003001 10637 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:19.003664 11408 heartbeater.cc:503] Master 127.24.66.126:40003 was elected leader, sending a full tablet report...
I20240502 14:01:19.005530 10637 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:19.006028 9696 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:19.006575 9564 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:19.007069 9828 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:19.008901 9038 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:19.010418 8890 catalog_manager.cc:5484] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:19.013949 9433 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:19.016585 9300 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:19.020464 9170 consensus_queue.cc:237] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:19.022794 9036 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:19.023870 9564 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:19.023926 9301 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:19.023954 11651 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:19.024385 9828 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:19.024849 9696 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:19.025756 11659 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:19.025966 11649 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:19.026101 11651 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:19.026234 10637 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:19.028857 9433 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:19.029331 11651 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:19.029754 11659 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:19.030387 9564 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:19.031073 9433 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:19.032413 9696 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:19.032842 9036 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:19.035059 9300 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:19.035681 8888 catalog_manager.cc:5484] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:19.036434 8882 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 3c597826f8094c2e81bebfb87d4c77b5 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 14:01:19.037304 9828 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
W20240502 14:01:19.044445 9106 consensus_peers.cc:485] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 -> Peer 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74:34421): Couldn't send request to peer 535b19a469c34efaaf49ca1cc2e1e5fc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3c597826f8094c2e81bebfb87d4c77b5. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:19.218967 11691 ts_tablet_manager.cc:902] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5: Initiating tablet copy from peer 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907)
I20240502 14:01:19.219846 11691 tablet_copy_client.cc:308] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:42907
I20240502 14:01:19.224293 9454 tablet_copy_service.cc:140] P 1a490f170c614ee4900ef16c3287c7de: Received BeginTabletCopySession request for tablet 9eef89d7e129492bab305a1b5009aace from peer 304274b88d234198a3892e38c05014a5 ({username='slave'} at 127.24.66.73:36691)
I20240502 14:01:19.224514 9454 tablet_copy_service.cc:161] P 1a490f170c614ee4900ef16c3287c7de: Beginning new tablet copy session on tablet 9eef89d7e129492bab305a1b5009aace from peer 304274b88d234198a3892e38c05014a5 at {username='slave'} at 127.24.66.73:36691: session id = 304274b88d234198a3892e38c05014a5-9eef89d7e129492bab305a1b5009aace
I20240502 14:01:19.225716 9454 tablet_copy_source_session.cc:205] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:19.229311 11691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eef89d7e129492bab305a1b5009aace. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:19.234979 11691 tablet_copy_client.cc:783] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:19.235193 11691 tablet_copy_client.cc:647] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:19.237036 11691 tablet_copy_client.cc:523] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:19.238726 11691 tablet_bootstrap.cc:492] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5: Bootstrap starting.
I20240502 14:01:19.244451 11691 log.cc:824] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5: Log is configured to *not* fsync() on all Append() calls
I20240502 14:01:19.273655 11691 tablet_bootstrap.cc:492] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:19.274104 11691 tablet_bootstrap.cc:492] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5: Bootstrap complete.
I20240502 14:01:19.274453 11691 ts_tablet_manager.cc:1367] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5: Time spent bootstrapping tablet: real 0.036s user 0.035s sys 0.000s
I20240502 14:01:19.277216 11691 raft_consensus.cc:348] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:19.277550 11691 raft_consensus.cc:729] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 304274b88d234198a3892e38c05014a5, State: Initialized, Role: LEARNER
I20240502 14:01:19.277915 11691 consensus_queue.cc:260] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:19.278860 11691 ts_tablet_manager.cc:1398] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
I20240502 14:01:19.280218 9454 tablet_copy_service.cc:342] P 1a490f170c614ee4900ef16c3287c7de: Request end of tablet copy session 304274b88d234198a3892e38c05014a5-9eef89d7e129492bab305a1b5009aace received from {username='slave'} at 127.24.66.73:36691
I20240502 14:01:19.280365 9454 tablet_copy_service.cc:433] P 1a490f170c614ee4900ef16c3287c7de: ending tablet copy session 304274b88d234198a3892e38c05014a5-9eef89d7e129492bab305a1b5009aace on tablet 9eef89d7e129492bab305a1b5009aace with peer 304274b88d234198a3892e38c05014a5
I20240502 14:01:19.284075 11691 ts_tablet_manager.cc:902] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5: Initiating tablet copy from peer ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:19.284580 11691 tablet_copy_client.cc:308] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.67:34749
I20240502 14:01:19.288805 9322 tablet_copy_service.cc:140] P ccfb5bbc1bb54793812d11170ac2d25f: Received BeginTabletCopySession request for tablet bfa2e6af0fd544fa8bfdaefedabb5e68 from peer 304274b88d234198a3892e38c05014a5 ({username='slave'} at 127.24.66.73:34459)
I20240502 14:01:19.288973 9322 tablet_copy_service.cc:161] P ccfb5bbc1bb54793812d11170ac2d25f: Beginning new tablet copy session on tablet bfa2e6af0fd544fa8bfdaefedabb5e68 from peer 304274b88d234198a3892e38c05014a5 at {username='slave'} at 127.24.66.73:34459: session id = 304274b88d234198a3892e38c05014a5-bfa2e6af0fd544fa8bfdaefedabb5e68
I20240502 14:01:19.290354 9322 tablet_copy_source_session.cc:205] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:19.291620 11691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfa2e6af0fd544fa8bfdaefedabb5e68. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:19.293622 11691 tablet_copy_client.cc:783] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:19.293815 11691 tablet_copy_client.cc:647] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:19.295221 11691 tablet_copy_client.cc:523] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:19.296320 11691 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5: Bootstrap starting.
I20240502 14:01:19.320077 11691 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:19.320461 11691 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5: Bootstrap complete.
I20240502 14:01:19.320708 11691 ts_tablet_manager.cc:1367] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5: Time spent bootstrapping tablet: real 0.024s user 0.016s sys 0.008s
I20240502 14:01:19.321379 11691 raft_consensus.cc:348] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:19.321549 11691 raft_consensus.cc:729] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 304274b88d234198a3892e38c05014a5, State: Initialized, Role: LEARNER
I20240502 14:01:19.321671 11691 consensus_queue.cc:260] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:19.322005 11691 ts_tablet_manager.cc:1398] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:19.322629 9322 tablet_copy_service.cc:342] P ccfb5bbc1bb54793812d11170ac2d25f: Request end of tablet copy session 304274b88d234198a3892e38c05014a5-bfa2e6af0fd544fa8bfdaefedabb5e68 received from {username='slave'} at 127.24.66.73:34459
I20240502 14:01:19.322734 9322 tablet_copy_service.cc:433] P ccfb5bbc1bb54793812d11170ac2d25f: ending tablet copy session 304274b88d234198a3892e38c05014a5-bfa2e6af0fd544fa8bfdaefedabb5e68 on tablet bfa2e6af0fd544fa8bfdaefedabb5e68 with peer 304274b88d234198a3892e38c05014a5
I20240502 14:01:19.335343 11734 ts_tablet_manager.cc:902] T f904eef49f9143cd957c8fa5c8ea25ea P 535b19a469c34efaaf49ca1cc2e1e5fc: Initiating tablet copy from peer bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459)
I20240502 14:01:19.336259 11734 tablet_copy_client.cc:308] T f904eef49f9143cd957c8fa5c8ea25ea P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.65:45459
I20240502 14:01:19.340291 9058 tablet_copy_service.cc:140] P bc9e4f2f668d48ea87d5aad4a2fb569d: Received BeginTabletCopySession request for tablet f904eef49f9143cd957c8fa5c8ea25ea from peer 535b19a469c34efaaf49ca1cc2e1e5fc ({username='slave'} at 127.24.66.74:45737)
I20240502 14:01:19.340499 9058 tablet_copy_service.cc:161] P bc9e4f2f668d48ea87d5aad4a2fb569d: Beginning new tablet copy session on tablet f904eef49f9143cd957c8fa5c8ea25ea from peer 535b19a469c34efaaf49ca1cc2e1e5fc at {username='slave'} at 127.24.66.74:45737: session id = 535b19a469c34efaaf49ca1cc2e1e5fc-f904eef49f9143cd957c8fa5c8ea25ea
I20240502 14:01:19.341786 9058 tablet_copy_source_session.cc:205] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:19.345331 11734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f904eef49f9143cd957c8fa5c8ea25ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:19.351647 11734 tablet_copy_client.cc:783] T f904eef49f9143cd957c8fa5c8ea25ea P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:19.351855 11734 tablet_copy_client.cc:647] T f904eef49f9143cd957c8fa5c8ea25ea P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:19.353933 11734 tablet_copy_client.cc:523] T f904eef49f9143cd957c8fa5c8ea25ea P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:19.355747 11734 tablet_bootstrap.cc:492] T f904eef49f9143cd957c8fa5c8ea25ea P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap starting.
I20240502 14:01:19.361667 11734 log.cc:824] T f904eef49f9143cd957c8fa5c8ea25ea P 535b19a469c34efaaf49ca1cc2e1e5fc: Log is configured to *not* fsync() on all Append() calls
I20240502 14:01:19.383334 11734 tablet_bootstrap.cc:492] T f904eef49f9143cd957c8fa5c8ea25ea P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:19.383723 11734 tablet_bootstrap.cc:492] T f904eef49f9143cd957c8fa5c8ea25ea P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap complete.
I20240502 14:01:19.384068 11734 ts_tablet_manager.cc:1367] T f904eef49f9143cd957c8fa5c8ea25ea P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent bootstrapping tablet: real 0.029s user 0.026s sys 0.000s
I20240502 14:01:19.386253 11734 raft_consensus.cc:348] T f904eef49f9143cd957c8fa5c8ea25ea P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:19.386546 11734 raft_consensus.cc:729] T f904eef49f9143cd957c8fa5c8ea25ea P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 535b19a469c34efaaf49ca1cc2e1e5fc, State: Initialized, Role: LEARNER
I20240502 14:01:19.386968 11734 consensus_queue.cc:260] T f904eef49f9143cd957c8fa5c8ea25ea P 535b19a469c34efaaf49ca1cc2e1e5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:19.387872 11734 ts_tablet_manager.cc:1398] T f904eef49f9143cd957c8fa5c8ea25ea P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent starting tablet: real 0.004s user 0.007s sys 0.000s
I20240502 14:01:19.388685 9058 tablet_copy_service.cc:342] P bc9e4f2f668d48ea87d5aad4a2fb569d: Request end of tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-f904eef49f9143cd957c8fa5c8ea25ea received from {username='slave'} at 127.24.66.74:45737
I20240502 14:01:19.388804 9058 tablet_copy_service.cc:433] P bc9e4f2f668d48ea87d5aad4a2fb569d: ending tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-f904eef49f9143cd957c8fa5c8ea25ea on tablet f904eef49f9143cd957c8fa5c8ea25ea with peer 535b19a469c34efaaf49ca1cc2e1e5fc
I20240502 14:01:19.399921 11691 ts_tablet_manager.cc:902] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5: Initiating tablet copy from peer 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:19.400452 11691 tablet_copy_client.cc:308] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.71:33679
I20240502 14:01:19.403824 9850 tablet_copy_service.cc:140] P 0488b0e4ed994db19d40aed8c15f945b: Received BeginTabletCopySession request for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a from peer 304274b88d234198a3892e38c05014a5 ({username='slave'} at 127.24.66.73:58115)
I20240502 14:01:19.404011 9850 tablet_copy_service.cc:161] P 0488b0e4ed994db19d40aed8c15f945b: Beginning new tablet copy session on tablet 97fa16722a1f4adcbf8a5dc3fdfab77a from peer 304274b88d234198a3892e38c05014a5 at {username='slave'} at 127.24.66.73:58115: session id = 304274b88d234198a3892e38c05014a5-97fa16722a1f4adcbf8a5dc3fdfab77a
I20240502 14:01:19.405392 9850 tablet_copy_source_session.cc:205] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:19.406522 11691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:19.408320 11691 tablet_copy_client.cc:783] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:19.408557 11691 tablet_copy_client.cc:647] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:19.410121 11691 tablet_copy_client.cc:523] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:19.411224 11691 tablet_bootstrap.cc:492] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5: Bootstrap starting.
I20240502 14:01:19.429988 11691 tablet_bootstrap.cc:492] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:19.430311 11691 tablet_bootstrap.cc:492] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5: Bootstrap complete.
I20240502 14:01:19.430481 11691 ts_tablet_manager.cc:1367] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5: Time spent bootstrapping tablet: real 0.019s user 0.018s sys 0.000s
I20240502 14:01:19.430931 11691 raft_consensus.cc:348] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:19.431074 11691 raft_consensus.cc:729] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 304274b88d234198a3892e38c05014a5, State: Initialized, Role: LEARNER
I20240502 14:01:19.431162 11691 consensus_queue.cc:260] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:19.431428 11691 ts_tablet_manager.cc:1398] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:19.431965 9850 tablet_copy_service.cc:342] P 0488b0e4ed994db19d40aed8c15f945b: Request end of tablet copy session 304274b88d234198a3892e38c05014a5-97fa16722a1f4adcbf8a5dc3fdfab77a received from {username='slave'} at 127.24.66.73:58115
I20240502 14:01:19.432060 9850 tablet_copy_service.cc:433] P 0488b0e4ed994db19d40aed8c15f945b: ending tablet copy session 304274b88d234198a3892e38c05014a5-97fa16722a1f4adcbf8a5dc3fdfab77a on tablet 97fa16722a1f4adcbf8a5dc3fdfab77a with peer 304274b88d234198a3892e38c05014a5
I20240502 14:01:19.469509 11748 ts_tablet_manager.cc:902] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7: Initiating tablet copy from peer 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907)
I20240502 14:01:19.470361 11748 tablet_copy_client.cc:308] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:42907
I20240502 14:01:19.474462 9454 tablet_copy_service.cc:140] P 1a490f170c614ee4900ef16c3287c7de: Received BeginTabletCopySession request for tablet 8c10fb9748c747c6a89066fa113cda3c from peer e01cf117ef5d446590c06dd1a6ab7ef7 ({username='slave'} at 127.24.66.72:40361)
I20240502 14:01:19.474598 9454 tablet_copy_service.cc:161] P 1a490f170c614ee4900ef16c3287c7de: Beginning new tablet copy session on tablet 8c10fb9748c747c6a89066fa113cda3c from peer e01cf117ef5d446590c06dd1a6ab7ef7 at {username='slave'} at 127.24.66.72:40361: session id = e01cf117ef5d446590c06dd1a6ab7ef7-8c10fb9748c747c6a89066fa113cda3c
I20240502 14:01:19.475384 9454 tablet_copy_source_session.cc:205] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:19.478662 11748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c10fb9748c747c6a89066fa113cda3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:19.485723 11748 tablet_copy_client.cc:783] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:19.485967 11748 tablet_copy_client.cc:647] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:19.487604 11748 tablet_copy_client.cc:523] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:19.489295 11748 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap starting.
I20240502 14:01:19.494720 11748 log.cc:824] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7: Log is configured to *not* fsync() on all Append() calls
I20240502 14:01:19.516731 11748 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:19.517124 11748 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap complete.
I20240502 14:01:19.517478 11748 ts_tablet_manager.cc:1367] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent bootstrapping tablet: real 0.028s user 0.023s sys 0.005s
I20240502 14:01:19.519940 11748 raft_consensus.cc:348] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:19.520239 11748 raft_consensus.cc:729] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e01cf117ef5d446590c06dd1a6ab7ef7, State: Initialized, Role: LEARNER
I20240502 14:01:19.520629 11748 consensus_queue.cc:260] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:19.521505 11748 ts_tablet_manager.cc:1398] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20240502 14:01:19.522285 9454 tablet_copy_service.cc:342] P 1a490f170c614ee4900ef16c3287c7de: Request end of tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-8c10fb9748c747c6a89066fa113cda3c received from {username='slave'} at 127.24.66.72:40361
I20240502 14:01:19.522380 9454 tablet_copy_service.cc:433] P 1a490f170c614ee4900ef16c3287c7de: ending tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-8c10fb9748c747c6a89066fa113cda3c on tablet 8c10fb9748c747c6a89066fa113cda3c with peer e01cf117ef5d446590c06dd1a6ab7ef7
I20240502 14:01:19.596364 11734 ts_tablet_manager.cc:902] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc: Initiating tablet copy from peer d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837)
I20240502 14:01:19.596871 11734 tablet_copy_client.cc:308] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.66:38837
I20240502 14:01:19.601207 9190 tablet_copy_service.cc:140] P d83af1bc823a4f2ca531f0d6825f8e16: Received BeginTabletCopySession request for tablet 3c597826f8094c2e81bebfb87d4c77b5 from peer 535b19a469c34efaaf49ca1cc2e1e5fc ({username='slave'} at 127.24.66.74:55643)
I20240502 14:01:19.601388 9190 tablet_copy_service.cc:161] P d83af1bc823a4f2ca531f0d6825f8e16: Beginning new tablet copy session on tablet 3c597826f8094c2e81bebfb87d4c77b5 from peer 535b19a469c34efaaf49ca1cc2e1e5fc at {username='slave'} at 127.24.66.74:55643: session id = 535b19a469c34efaaf49ca1cc2e1e5fc-3c597826f8094c2e81bebfb87d4c77b5
I20240502 14:01:19.602989 9190 tablet_copy_source_session.cc:205] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:19.604441 11734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c597826f8094c2e81bebfb87d4c77b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:19.606509 11734 tablet_copy_client.cc:783] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:19.606689 11734 tablet_copy_client.cc:647] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:19.608175 11734 tablet_copy_client.cc:523] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:19.609656 11734 tablet_bootstrap.cc:492] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap starting.
I20240502 14:01:19.628248 11100 raft_consensus.cc:1203] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 14:01:19.629974 11734 tablet_bootstrap.cc:492] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:19.630285 11734 tablet_bootstrap.cc:492] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap complete.
I20240502 14:01:19.630484 11734 ts_tablet_manager.cc:1367] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent bootstrapping tablet: real 0.021s user 0.019s sys 0.000s
I20240502 14:01:19.631111 11734 raft_consensus.cc:348] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:19.631269 11734 raft_consensus.cc:729] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 535b19a469c34efaaf49ca1cc2e1e5fc, State: Initialized, Role: LEARNER
I20240502 14:01:19.631356 11734 consensus_queue.cc:260] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:19.631640 11734 ts_tablet_manager.cc:1398] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:19.632293 9190 tablet_copy_service.cc:342] P d83af1bc823a4f2ca531f0d6825f8e16: Request end of tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-3c597826f8094c2e81bebfb87d4c77b5 received from {username='slave'} at 127.24.66.74:55643
I20240502 14:01:19.632436 9190 tablet_copy_service.cc:433] P d83af1bc823a4f2ca531f0d6825f8e16: ending tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-3c597826f8094c2e81bebfb87d4c77b5 on tablet 3c597826f8094c2e81bebfb87d4c77b5 with peer 535b19a469c34efaaf49ca1cc2e1e5fc
I20240502 14:01:19.776476 11100 raft_consensus.cc:1203] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 14:01:19.796741 11362 raft_consensus.cc:1203] T f904eef49f9143cd957c8fa5c8ea25ea P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 14:01:20.001646 11100 raft_consensus.cc:1203] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 14:01:20.005970 11231 raft_consensus.cc:1203] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 14:01:20.095800 11362 raft_consensus.cc:1203] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 14:01:20.113090 11585 raft_consensus.cc:1050] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de: attempting to promote NON_VOTER 304274b88d234198a3892e38c05014a5 to VOTER
I20240502 14:01:20.113821 11585 consensus_queue.cc:237] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:20.115840 9830 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.115897 9697 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.115947 9564 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.116226 9300 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.116364 11859 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.116513 11100 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.116730 9036 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.116925 11859 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.117187 9170 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.117710 11861 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.118057 11585 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.118407 11585 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.118556 11859 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.119634 11559 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.121846 9300 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.122701 9830 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.123041 9036 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.125341 8895 catalog_manager.cc:5484] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.125231 11100 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.121393 11858 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.129841 9564 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.129879 9697 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.131108 9170 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.134909 9433 consensus_queue.cc:237] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:20.136516 9830 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.136883 11100 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.137290 9170 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.137357 9036 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.137847 11859 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.138068 11858 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.138245 11585 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.139940 11858 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.140290 9300 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.140748 11858 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.142153 9697 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.142030 11858 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.142611 11861 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.143162 9300 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.143714 9696 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.143755 9830 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.145778 8882 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 9eef89d7e129492bab305a1b5009aace with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:20.145876 9170 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.146252 11100 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.146693 9036 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.148680 8894 catalog_manager.cc:5484] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de reported cstate change: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:20.152148 9544 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eef89d7e129492bab305a1b5009aace with delete_type TABLET_DATA_TOMBSTONED (TS d0f45862d7f14ce1976c1a37842f25ca not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:46242
I20240502 14:01:20.153625 11874 tablet_replica.cc:330] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca: stopping tablet replica
I20240502 14:01:20.153990 11874 raft_consensus.cc:2229] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:20.154287 11874 raft_consensus.cc:2258] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:20.155784 11874 ts_tablet_manager.cc:1875] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:20.157889 11874 ts_tablet_manager.cc:1888] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 14:01:20.157969 11874 log.cc:1196] T 9eef89d7e129492bab305a1b5009aace P d0f45862d7f14ce1976c1a37842f25ca: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal/wals/9eef89d7e129492bab305a1b5009aace
I20240502 14:01:20.158555 8881 catalog_manager.cc:4830] TS d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375): tablet 9eef89d7e129492bab305a1b5009aace (table rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]) successfully deleted
I20240502 14:01:20.381238 11581 raft_consensus.cc:1050] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f: attempting to promote NON_VOTER 304274b88d234198a3892e38c05014a5 to VOTER
I20240502 14:01:20.381724 11581 consensus_queue.cc:237] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:20.383438 11100 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.383617 9830 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.383612 9564 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.383687 9434 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.384162 11907 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.384559 9168 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.384599 9036 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.384603 11581 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.385076 11581 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.385200 9696 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.385581 11907 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.386673 11909 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.387079 11908 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.387349 11910 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.388777 11581 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.390214 9566 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.390942 9830 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.391319 9036 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.392450 11100 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.392974 9168 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.393144 8888 catalog_manager.cc:5484] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:20.394600 9434 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.398798 9696 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.402693 9300 consensus_queue.cc:237] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:20.404626 9566 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.404917 11100 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.404976 9696 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.405208 9168 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.405347 11798 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.405584 11908 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.405725 11798 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.405891 11581 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.406617 9036 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.407495 9830 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.407847 11908 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.408126 11798 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.408622 11908 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.409653 9168 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.410104 9038 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.410859 9830 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.413504 8890 catalog_manager.cc:5484] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.413738 8880 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet bfa2e6af0fd544fa8bfdaefedabb5e68 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:20.414047 9696 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.415017 9566 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.416996 11815 raft_consensus.cc:1050] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d: attempting to promote NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc to VOTER
I20240502 14:01:20.417639 11815 consensus_queue.cc:237] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:20.418958 11100 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.419543 9412 tablet_service.cc:1514] Processing DeleteTablet for tablet bfa2e6af0fd544fa8bfdaefedabb5e68 with delete_type TABLET_DATA_TOMBSTONED (TS 1a490f170c614ee4900ef16c3287c7de not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:33738
I20240502 14:01:20.419746 9300 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.419960 9828 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.420401 11921 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.420799 11921 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.421851 11923 tablet_replica.cc:330] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de: stopping tablet replica
I20240502 14:01:20.422168 11923 raft_consensus.cc:2229] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:20.422449 11923 raft_consensus.cc:2258] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:20.424248 11923 ts_tablet_manager.cc:1875] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:20.424271 11362 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.424578 9696 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.425034 11922 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.425148 9168 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.425431 11922 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.425774 11920 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.428485 9434 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.428552 11923 ts_tablet_manager.cc:1888] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 14:01:20.428682 11923 log.cc:1196] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/bfa2e6af0fd544fa8bfdaefedabb5e68
I20240502 14:01:20.428964 9566 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.428943 11921 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.429440 11920 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.430137 8882 catalog_manager.cc:4830] TS 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907): tablet bfa2e6af0fd544fa8bfdaefedabb5e68 (table rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]) successfully deleted
I20240502 14:01:20.432178 11815 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.433789 9434 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.434080 11362 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.436167 9696 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.436259 9300 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.438426 8889 catalog_manager.cc:5484] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d reported cstate change: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:20.440526 9828 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.443647 9566 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.446633 9168 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.449223 9036 consensus_queue.cc:237] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:20.450954 9168 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.450954 9828 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.451326 11815 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.451581 9566 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.452049 9696 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.452320 11362 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.452215 9300 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.452289 11921 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.453038 11815 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.453172 11918 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.453320 11922 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.453528 11920 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.455467 11922 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.458153 9828 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.458206 9300 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.459110 8896 catalog_manager.cc:5484] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d reported cstate change: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:20.460970 8882 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f904eef49f9143cd957c8fa5c8ea25ea with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:20.461501 11362 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.462273 9566 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.463862 9412 tablet_service.cc:1514] Processing DeleteTablet for tablet f904eef49f9143cd957c8fa5c8ea25ea with delete_type TABLET_DATA_TOMBSTONED (TS 1a490f170c614ee4900ef16c3287c7de not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:33738
I20240502 14:01:20.464092 11923 tablet_replica.cc:330] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de: stopping tablet replica
I20240502 14:01:20.464208 11923 raft_consensus.cc:2229] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:20.464433 11923 raft_consensus.cc:2258] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:20.465363 11923 ts_tablet_manager.cc:1875] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:20.465395 9696 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.467496 11923 ts_tablet_manager.cc:1888] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 14:01:20.467571 11923 log.cc:1196] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/f904eef49f9143cd957c8fa5c8ea25ea
I20240502 14:01:20.467573 9168 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.470808 8882 catalog_manager.cc:4830] TS 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907): tablet f904eef49f9143cd957c8fa5c8ea25ea (table rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]) successfully deleted
I20240502 14:01:20.504979 9434 consensus_queue.cc:237] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:20.507087 9168 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:20.507306 9696 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:20.507728 9566 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:20.507833 11861 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:20.507942 9828 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:20.507966 9300 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:20.508400 11585 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:20.508772 11585 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:20.509539 11585 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:20.510264 11858 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:20.510986 11858 raft_consensus.cc:1050] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de: attempting to promote NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 to VOTER
W20240502 14:01:20.511341 11861 consensus_queue.cc:1159] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "efa836edab8d4853a7ca47a23117e288" responder_term: 1 status { last_received { term: 1 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 1 index: 5 } }
I20240502 14:01:20.511576 11858 consensus_queue.cc:237] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } }
I20240502 14:01:20.512276 9036 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:20.513855 11231 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.513865 9168 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.514008 11861 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:20.514415 9036 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.514350 11861 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:20.514912 11945 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.515478 11944 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.515555 9036 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:20.517323 9828 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.517486 9564 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:20.517338 9830 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:20.518097 9566 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.519352 11860 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.519963 9300 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.520696 11860 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.520669 9697 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:20.520774 8895 catalog_manager.cc:5484] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:20.521198 11860 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.521852 9170 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:20.522735 9696 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.525029 11859 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.525559 11859 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.525766 9300 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:20.529178 11858 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:20.529451 9434 consensus_queue.cc:237] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:20.531157 9038 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:20.531922 9696 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:20.531916 11560 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:20.531819 9168 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:20.532147 9038 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
W20240502 14:01:20.532424 9370 consensus_peers.cc:485] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de -> Peer e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72:35005): Couldn't send request to peer e01cf117ef5d446590c06dd1a6ab7ef7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b1548ca1bcd24d288d30117baea53c12. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:20.532541 11560 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:20.533474 9170 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:20.534543 11231 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:20.534835 9566 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:20.537501 11861 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:20.537736 9564 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:20.538340 11746 raft_consensus.cc:1050] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b: attempting to promote NON_VOTER 304274b88d234198a3892e38c05014a5 to VOTER
I20240502 14:01:20.538271 9696 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:20.538906 11746 consensus_queue.cc:237] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:20.539880 11585 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:20.541237 9300 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:20.541518 9830 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:20.541814 9828 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:20.541998 8896 catalog_manager.cc:5484] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d reported cstate change: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:20.542253 9697 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:20.542657 9170 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:20.542944 8882 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet b1548ca1bcd24d288d30117baea53c12 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 14:01:20.544106 9300 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.544543 11100 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.544544 11614 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.545284 9696 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.545361 11614 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.545696 11826 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.545805 11861 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:20.546128 11861 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:20.546873 11859 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:20.547904 9302 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:20.548789 9434 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.549320 11826 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.549857 9566 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:20.550297 9828 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:20.553928 9170 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.554092 9038 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.554318 11614 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.554790 9036 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:20.556356 11614 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.558096 8888 catalog_manager.cc:5484] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de reported cstate change: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:20.559648 9564 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.560173 11969 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.562057 9434 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.560173 11614 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20240502 14:01:20.565889 9566 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.566480 9300 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.566550 8889 catalog_manager.cc:5484] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de reported cstate change: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.570039 9696 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.570258 11791 raft_consensus.cc:1050] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16: attempting to promote NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc to VOTER
I20240502 14:01:20.570880 11791 consensus_queue.cc:237] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:20.573030 11362 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.573444 9433 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.574002 9564 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.573976 11983 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.574380 9300 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:20.574844 9696 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.575016 9170 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.575335 11651 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.575709 11651 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.576570 9828 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.577488 11100 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.577466 9038 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.579625 11659 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.579957 11651 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.582882 9830 consensus_queue.cc:237] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:20.583077 9434 consensus_queue.cc:237] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } }
I20240502 14:01:20.584767 9696 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.584918 9301 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.585342 9168 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.585444 9036 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.586033 11231 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.586223 9696 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.586266 9038 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.586525 11978 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.586630 11560 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.586670 11969 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.586968 11560 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.587045 11969 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.587574 9170 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.587931 9301 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.587927 11858 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.588232 11858 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.588706 11859 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.590575 11100 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.590636 11965 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.591017 9300 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.591058 11965 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.591596 11791 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.591446 11946 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:20.592558 9036 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:20.592948 11659 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:20.594156 9434 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.594235 9035 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:20.595436 11791 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.596784 9302 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:20.596774 9168 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.598062 8889 catalog_manager.cc:5484] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d reported cstate change: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:20.596784 9300 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.596303 11965 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.599295 8882 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 8c10fb9748c747c6a89066fa113cda3c with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:20.600134 9830 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.600003 11362 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.600486 11231 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:20.601976 11858 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.602955 9696 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:20.603025 8895 catalog_manager.cc:5484] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc reported cstate change: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.603963 9697 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.604406 9830 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:20.606297 9544 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c10fb9748c747c6a89066fa113cda3c with delete_type TABLET_DATA_TOMBSTONED (TS d0f45862d7f14ce1976c1a37842f25ca not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:46242
I20240502 14:01:20.606547 11874 tablet_replica.cc:330] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca: stopping tablet replica
I20240502 14:01:20.606698 11874 raft_consensus.cc:2229] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:20.606940 11874 raft_consensus.cc:2258] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:20.607197 9434 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.607832 11874 ts_tablet_manager.cc:1875] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:20.608036 9168 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:20.609473 9301 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.610858 11874 ts_tablet_manager.cc:1888] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 14:01:20.610929 11874 log.cc:1196] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal/wals/8c10fb9748c747c6a89066fa113cda3c
I20240502 14:01:20.611559 11964 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.612218 8880 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:20.612360 8881 catalog_manager.cc:4830] TS d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375): tablet 8c10fb9748c747c6a89066fa113cda3c (table rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]) successfully deleted
I20240502 14:01:20.613004 8893 catalog_manager.cc:5484] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:20.614794 11100 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.614914 9697 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.614930 9038 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.617305 9544 tablet_service.cc:1514] Processing DeleteTablet for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a with delete_type TABLET_DATA_TOMBSTONED (TS d0f45862d7f14ce1976c1a37842f25ca not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:46242
I20240502 14:01:20.619196 9434 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:20.621001 11874 tablet_replica.cc:330] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d0f45862d7f14ce1976c1a37842f25ca: stopping tablet replica
I20240502 14:01:20.621284 11874 raft_consensus.cc:2229] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:20.621918 11874 raft_consensus.cc:2258] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:20.623199 9564 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.625181 9830 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.629184 11874 ts_tablet_manager.cc:1875] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d0f45862d7f14ce1976c1a37842f25ca: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:20.631188 11874 ts_tablet_manager.cc:1888] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d0f45862d7f14ce1976c1a37842f25ca: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 14:01:20.631276 11874 log.cc:1196] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d0f45862d7f14ce1976c1a37842f25ca: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal/wals/97fa16722a1f4adcbf8a5dc3fdfab77a
I20240502 14:01:20.631773 8881 catalog_manager.cc:4830] TS d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375): tablet 97fa16722a1f4adcbf8a5dc3fdfab77a (table rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]) successfully deleted
I20240502 14:01:20.633459 9036 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.637301 9168 consensus_queue.cc:237] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:20.639129 9828 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.639361 9696 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.639351 11362 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.639663 9564 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.639904 9301 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.640122 11659 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.640475 11983 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.640630 11659 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.641014 11651 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.642136 11982 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.643349 11651 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.644152 9036 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:20.645022 11791 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:20.645233 9301 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.646392 11362 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.646757 8889 catalog_manager.cc:5484] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:20.648900 9036 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.650728 9696 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.652740 8882 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 3c597826f8094c2e81bebfb87d4c77b5 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:20.653075 9830 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.657123 9412 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c597826f8094c2e81bebfb87d4c77b5 with delete_type TABLET_DATA_TOMBSTONED (TS 1a490f170c614ee4900ef16c3287c7de not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:33738
I20240502 14:01:20.657382 11923 tablet_replica.cc:330] T 3c597826f8094c2e81bebfb87d4c77b5 P 1a490f170c614ee4900ef16c3287c7de: stopping tablet replica
I20240502 14:01:20.657531 11923 raft_consensus.cc:2229] T 3c597826f8094c2e81bebfb87d4c77b5 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:20.657714 11923 raft_consensus.cc:2258] T 3c597826f8094c2e81bebfb87d4c77b5 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:20.658854 11923 ts_tablet_manager.cc:1875] T 3c597826f8094c2e81bebfb87d4c77b5 P 1a490f170c614ee4900ef16c3287c7de: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:20.661288 11923 ts_tablet_manager.cc:1888] T 3c597826f8094c2e81bebfb87d4c77b5 P 1a490f170c614ee4900ef16c3287c7de: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 14:01:20.661361 11923 log.cc:1196] T 3c597826f8094c2e81bebfb87d4c77b5 P 1a490f170c614ee4900ef16c3287c7de: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/3c597826f8094c2e81bebfb87d4c77b5
I20240502 14:01:20.661309 9564 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:20.661936 8882 catalog_manager.cc:4830] TS 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907): tablet 3c597826f8094c2e81bebfb87d4c77b5 (table rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]) successfully deleted
I20240502 14:01:20.886564 9170 consensus_queue.cc:237] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:20.888446 9434 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:20.888573 9036 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:20.888594 9301 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:20.888612 9696 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:20.888877 9564 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:20.888963 9828 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:20.889058 11990 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:20.889325 11982 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:20.889461 11990 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:20.889873 11984 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:20.891469 11990 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:20.891762 11659 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:20.893342 11659 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:20.894284 9434 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:20.894392 9696 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:20.894475 9828 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:20.895913 9036 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:20.896206 9301 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:20.898016 8892 catalog_manager.cc:5484] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:20.899284 9566 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:20.906070 9170 consensus_queue.cc:237] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:20.907981 9301 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:20.908435 11651 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:20.909977 9036 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:20.910622 9566 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:20.910373 11791 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:20.910986 9434 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:20.911098 11984 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:20.911389 11990 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:20.912496 9696 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:20.912541 9828 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:20.913054 11651 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:20.913383 11651 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:20.914855 11983 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:20.915784 9696 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:20.916469 9434 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:20.917534 9566 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:20.918116 8882 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 29eec0daa5d34e40a4ad82607b7ded90 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 14:01:20.918768 8896 catalog_manager.cc:5484] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:20.918984 9301 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:20.921322 9828 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:20.922116 9036 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:20.926708 9301 consensus_queue.cc:237] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
W20240502 14:01:20.928561 9106 consensus_peers.cc:485] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 -> Peer e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72:35005): Couldn't send request to peer e01cf117ef5d446590c06dd1a6ab7ef7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 29eec0daa5d34e40a4ad82607b7ded90. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:20.929257 9828 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:20.929256 9170 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:20.929445 9696 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:20.929623 11907 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:20.930003 11907 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:20.930133 11910 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:20.930173 9036 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:20.931087 9566 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:20.931244 11907 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:20.931558 11907 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:20.932094 9434 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:20.932811 11907 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:20.933701 11907 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:20.934195 9696 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:20.934506 9036 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:20.936419 9170 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:20.937294 9828 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:20.937767 8893 catalog_manager.cc:5484] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:20.939049 9566 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:20.941900 9433 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:20.944844 9301 consensus_queue.cc:237] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:20.946321 12045 ts_tablet_manager.cc:902] T b1548ca1bcd24d288d30117baea53c12 P e01cf117ef5d446590c06dd1a6ab7ef7: Initiating tablet copy from peer 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907)
I20240502 14:01:20.947291 12045 tablet_copy_client.cc:308] T b1548ca1bcd24d288d30117baea53c12 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:42907
I20240502 14:01:20.947770 9566 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:20.948163 9696 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:20.948129 11910 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:20.948179 9828 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:20.948545 9454 tablet_copy_service.cc:140] P 1a490f170c614ee4900ef16c3287c7de: Received BeginTabletCopySession request for tablet b1548ca1bcd24d288d30117baea53c12 from peer e01cf117ef5d446590c06dd1a6ab7ef7 ({username='slave'} at 127.24.66.72:40361)
I20240502 14:01:20.948588 12048 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:20.948647 9454 tablet_copy_service.cc:161] P 1a490f170c614ee4900ef16c3287c7de: Beginning new tablet copy session on tablet b1548ca1bcd24d288d30117baea53c12 from peer e01cf117ef5d446590c06dd1a6ab7ef7 at {username='slave'} at 127.24.66.72:40361: session id = e01cf117ef5d446590c06dd1a6ab7ef7-b1548ca1bcd24d288d30117baea53c12
I20240502 14:01:20.949169 9036 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:20.949265 9170 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:20.949594 9433 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:20.950724 12033 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:20.951050 11907 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:20.951258 12033 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:20.949618 9454 tablet_copy_source_session.cc:205] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:20.951413 11910 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:20.952395 12045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1548ca1bcd24d288d30117baea53c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:20.952571 11910 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:20.953935 9696 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:20.955093 8880 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 50451fdbd43b491fa701daac5de5e1c8 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 14:01:20.955404 9170 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:20.955417 9828 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:20.955567 9566 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:20.956944 9036 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:20.958370 8889 catalog_manager.cc:5484] T 50451fdbd43b491fa701daac5de5e1c8 P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:20.959839 9433 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:20.961031 12045 tablet_copy_client.cc:783] T b1548ca1bcd24d288d30117baea53c12 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:20.961210 12045 tablet_copy_client.cc:647] T b1548ca1bcd24d288d30117baea53c12 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 1 WAL segments...
W20240502 14:01:20.963356 9238 consensus_peers.cc:485] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f -> Peer e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72:35005): Couldn't send request to peer e01cf117ef5d446590c06dd1a6ab7ef7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 50451fdbd43b491fa701daac5de5e1c8. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:20.966872 12045 tablet_copy_client.cc:523] T b1548ca1bcd24d288d30117baea53c12 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:20.967343 9433 consensus_queue.cc:237] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:20.968493 12045 tablet_bootstrap.cc:492] T b1548ca1bcd24d288d30117baea53c12 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap starting.
I20240502 14:01:20.969226 9170 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:20.969300 9301 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:20.969442 9828 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:20.969864 11946 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:20.970052 11560 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:20.970221 11947 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:20.971050 9696 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:20.971518 11560 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:20.973224 9036 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:20.973867 11946 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:20.973968 9566 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:20.974222 11947 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:20.974826 9696 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:20.975677 11859 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:20.976630 9301 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:20.977635 9566 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:20.978698 8889 catalog_manager.cc:5484] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288 reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:20.981647 9828 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:20.981705 9170 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:20.987195 9433 consensus_queue.cc:237] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:20.989974 9300 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:20.990859 11859 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:20.992470 9170 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:20.992782 9828 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:20.992882 11859 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:20.993234 9696 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:20.993849 11947 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:20.994136 11947 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:20.998492 9566 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240502 14:01:20.998620 9370 consensus_peers.cc:485] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de -> Peer 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74:34421): Couldn't send request to peer 535b19a469c34efaaf49ca1cc2e1e5fc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5f84e791335948e180f5575f8174efd1. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:20.998695 11859 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:20.998996 11585 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:20.999950 9170 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:21.000174 9566 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:21.001291 8882 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 5f84e791335948e180f5575f8174efd1 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 14:01:21.001401 9696 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:21.001572 9828 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:21.003221 8896 catalog_manager.cc:5484] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca reported cstate change: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:21.011111 9038 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:21.015105 11945 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:21.015998 12045 tablet_bootstrap.cc:492] T b1548ca1bcd24d288d30117baea53c12 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:21.016418 12045 tablet_bootstrap.cc:492] T b1548ca1bcd24d288d30117baea53c12 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap complete.
I20240502 14:01:21.016212 9038 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:21.016801 12045 ts_tablet_manager.cc:1367] T b1548ca1bcd24d288d30117baea53c12 P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent bootstrapping tablet: real 0.048s user 0.030s sys 0.001s
I20240502 14:01:21.016260 9300 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:21.017614 12045 raft_consensus.cc:348] T b1548ca1bcd24d288d30117baea53c12 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:21.018347 12045 raft_consensus.cc:729] T b1548ca1bcd24d288d30117baea53c12 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e01cf117ef5d446590c06dd1a6ab7ef7, State: Initialized, Role: LEARNER
I20240502 14:01:21.018540 12045 consensus_queue.cc:260] T b1548ca1bcd24d288d30117baea53c12 P e01cf117ef5d446590c06dd1a6ab7ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:21.019882 12045 ts_tablet_manager.cc:1398] T b1548ca1bcd24d288d30117baea53c12 P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240502 14:01:21.022110 9454 tablet_copy_service.cc:342] P 1a490f170c614ee4900ef16c3287c7de: Request end of tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-b1548ca1bcd24d288d30117baea53c12 received from {username='slave'} at 127.24.66.72:40361
I20240502 14:01:21.022213 9454 tablet_copy_service.cc:433] P 1a490f170c614ee4900ef16c3287c7de: ending tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-b1548ca1bcd24d288d30117baea53c12 on tablet b1548ca1bcd24d288d30117baea53c12 with peer e01cf117ef5d446590c06dd1a6ab7ef7
I20240502 14:01:21.018077 9038 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:21.032733 9168 consensus_queue.cc:237] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:21.034952 9433 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:21.035180 9828 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:21.035387 11983 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:21.035799 11983 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:21.038070 9696 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:21.038409 9566 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:21.038568 11983 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:21.038823 11791 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:21.039175 9038 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:21.039933 9300 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:21.040057 11791 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:21.040376 11983 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:21.042477 11983 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:21.043092 9038 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:21.045344 9566 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:21.045827 8889 catalog_manager.cc:5484] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:21.050632 9433 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:21.052026 9828 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:21.054047 9168 consensus_queue.cc:237] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:21.056524 9566 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:21.056521 9830 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:21.056917 11791 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:21.057108 11983 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:21.058959 9036 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240502 14:01:21.058972 9106 consensus_peers.cc:485] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 -> Peer e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72:35005): Couldn't send request to peer e01cf117ef5d446590c06dd1a6ab7ef7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ada4c2cc1749411d8f41f882b7c64b07. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:21.059592 11651 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:21.059849 9697 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:21.060884 11983 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:21.061049 9433 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:21.061618 11659 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:21.063066 9697 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:21.070338 11659 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:21.072430 8882 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet ada4c2cc1749411d8f41f882b7c64b07 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 14:01:21.073267 9830 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:21.073264 9564 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:21.074954 9433 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:21.076907 8892 catalog_manager.cc:5484] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:21.079835 9301 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:21.080758 11990 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:21.081336 9036 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:21.081735 9301 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:21.082980 9301 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:21.091428 9697 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:21.341428 9830 consensus_queue.cc:237] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:21.343302 9301 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:21.343412 9697 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:21.343720 9564 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:21.343786 9168 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:21.343807 9036 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:21.344198 12107 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:21.344436 9434 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:21.344434 12116 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:21.344671 12108 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:21.344843 12107 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:21.344985 11978 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:21.345853 12116 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:21.348093 12107 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:21.348686 9301 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:21.348982 9036 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:21.350003 9697 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:21.350621 9564 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:21.352299 8892 catalog_manager.cc:5484] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:21.355096 9434 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:21.359840 9830 consensus_queue.cc:237] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:21.361037 9168 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:21.362758 9564 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:21.362879 9301 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:21.363297 12108 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:21.363402 9697 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:21.363617 9170 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:21.363673 12108 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:21.364398 9434 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:21.365682 9036 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:21.365757 12130 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:21.365938 12108 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:21.366183 12116 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:21.367708 12130 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:21.369982 12107 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:21.370371 9434 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:21.371281 9564 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:21.372247 8880 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 655c86a7f52a40c68c3e95cb12286c29 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 14:01:21.372872 9170 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:21.372903 9036 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:21.374051 9301 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:21.374954 8890 catalog_manager.cc:5484] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de reported cstate change: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
W20240502 14:01:21.375380 9766 consensus_peers.cc:485] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b -> Peer 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74:34421): Couldn't send request to peer 535b19a469c34efaaf49ca1cc2e1e5fc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 655c86a7f52a40c68c3e95cb12286c29. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:21.378245 9697 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:21.381852 12045 ts_tablet_manager.cc:902] T 50451fdbd43b491fa701daac5de5e1c8 P e01cf117ef5d446590c06dd1a6ab7ef7: Initiating tablet copy from peer ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:21.382367 12045 tablet_copy_client.cc:308] T 50451fdbd43b491fa701daac5de5e1c8 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.67:34749
I20240502 14:01:21.387300 9322 tablet_copy_service.cc:140] P ccfb5bbc1bb54793812d11170ac2d25f: Received BeginTabletCopySession request for tablet 50451fdbd43b491fa701daac5de5e1c8 from peer e01cf117ef5d446590c06dd1a6ab7ef7 ({username='slave'} at 127.24.66.72:33161)
I20240502 14:01:21.387429 9322 tablet_copy_service.cc:161] P ccfb5bbc1bb54793812d11170ac2d25f: Beginning new tablet copy session on tablet 50451fdbd43b491fa701daac5de5e1c8 from peer e01cf117ef5d446590c06dd1a6ab7ef7 at {username='slave'} at 127.24.66.72:33161: session id = e01cf117ef5d446590c06dd1a6ab7ef7-50451fdbd43b491fa701daac5de5e1c8
I20240502 14:01:21.388185 9322 tablet_copy_source_session.cc:205] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:21.389123 12045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50451fdbd43b491fa701daac5de5e1c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:21.391413 12045 tablet_copy_client.cc:783] T 50451fdbd43b491fa701daac5de5e1c8 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:21.391577 12045 tablet_copy_client.cc:647] T 50451fdbd43b491fa701daac5de5e1c8 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:21.392903 12045 tablet_copy_client.cc:523] T 50451fdbd43b491fa701daac5de5e1c8 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:21.393925 12045 tablet_bootstrap.cc:492] T 50451fdbd43b491fa701daac5de5e1c8 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap starting.
I20240502 14:01:21.407128 12153 ts_tablet_manager.cc:902] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7: Initiating tablet copy from peer d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837)
I20240502 14:01:21.408073 12153 tablet_copy_client.cc:308] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.66:38837
I20240502 14:01:21.413194 9190 tablet_copy_service.cc:140] P d83af1bc823a4f2ca531f0d6825f8e16: Received BeginTabletCopySession request for tablet 29eec0daa5d34e40a4ad82607b7ded90 from peer e01cf117ef5d446590c06dd1a6ab7ef7 ({username='slave'} at 127.24.66.72:37077)
I20240502 14:01:21.413317 9190 tablet_copy_service.cc:161] P d83af1bc823a4f2ca531f0d6825f8e16: Beginning new tablet copy session on tablet 29eec0daa5d34e40a4ad82607b7ded90 from peer e01cf117ef5d446590c06dd1a6ab7ef7 at {username='slave'} at 127.24.66.72:37077: session id = e01cf117ef5d446590c06dd1a6ab7ef7-29eec0daa5d34e40a4ad82607b7ded90
I20240502 14:01:21.414369 9190 tablet_copy_source_session.cc:205] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:21.414809 12158 ts_tablet_manager.cc:902] T 5f84e791335948e180f5575f8174efd1 P 535b19a469c34efaaf49ca1cc2e1e5fc: Initiating tablet copy from peer 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907)
I20240502 14:01:21.416100 12153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29eec0daa5d34e40a4ad82607b7ded90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:21.417644 12158 tablet_copy_client.cc:308] T 5f84e791335948e180f5575f8174efd1 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:42907
I20240502 14:01:21.418534 12153 tablet_copy_client.cc:783] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:21.418685 12153 tablet_copy_client.cc:647] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:21.421444 12045 tablet_bootstrap.cc:492] T 50451fdbd43b491fa701daac5de5e1c8 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:21.421525 12153 tablet_copy_client.cc:523] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:21.421851 12045 tablet_bootstrap.cc:492] T 50451fdbd43b491fa701daac5de5e1c8 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap complete.
I20240502 14:01:21.422025 12045 ts_tablet_manager.cc:1367] T 50451fdbd43b491fa701daac5de5e1c8 P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent bootstrapping tablet: real 0.028s user 0.026s sys 0.000s
I20240502 14:01:21.422506 12045 raft_consensus.cc:348] T 50451fdbd43b491fa701daac5de5e1c8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:21.422638 12045 raft_consensus.cc:729] T 50451fdbd43b491fa701daac5de5e1c8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e01cf117ef5d446590c06dd1a6ab7ef7, State: Initialized, Role: LEARNER
I20240502 14:01:21.422879 12153 tablet_bootstrap.cc:492] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap starting.
I20240502 14:01:21.423027 9454 tablet_copy_service.cc:140] P 1a490f170c614ee4900ef16c3287c7de: Received BeginTabletCopySession request for tablet 5f84e791335948e180f5575f8174efd1 from peer 535b19a469c34efaaf49ca1cc2e1e5fc ({username='slave'} at 127.24.66.74:50277)
I20240502 14:01:21.423130 9454 tablet_copy_service.cc:161] P 1a490f170c614ee4900ef16c3287c7de: Beginning new tablet copy session on tablet 5f84e791335948e180f5575f8174efd1 from peer 535b19a469c34efaaf49ca1cc2e1e5fc at {username='slave'} at 127.24.66.74:50277: session id = 535b19a469c34efaaf49ca1cc2e1e5fc-5f84e791335948e180f5575f8174efd1
I20240502 14:01:21.423077 12045 consensus_queue.cc:260] T 50451fdbd43b491fa701daac5de5e1c8 P e01cf117ef5d446590c06dd1a6ab7ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:21.423552 12045 ts_tablet_manager.cc:1398] T 50451fdbd43b491fa701daac5de5e1c8 P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:21.424031 9322 tablet_copy_service.cc:342] P ccfb5bbc1bb54793812d11170ac2d25f: Request end of tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-50451fdbd43b491fa701daac5de5e1c8 received from {username='slave'} at 127.24.66.72:33161
I20240502 14:01:21.424106 9322 tablet_copy_service.cc:433] P ccfb5bbc1bb54793812d11170ac2d25f: ending tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-50451fdbd43b491fa701daac5de5e1c8 on tablet 50451fdbd43b491fa701daac5de5e1c8 with peer e01cf117ef5d446590c06dd1a6ab7ef7
I20240502 14:01:21.424090 9454 tablet_copy_source_session.cc:205] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:21.426383 12158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f84e791335948e180f5575f8174efd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:21.429339 12158 tablet_copy_client.cc:783] T 5f84e791335948e180f5575f8174efd1 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:21.429564 12158 tablet_copy_client.cc:647] T 5f84e791335948e180f5575f8174efd1 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:21.433586 12158 tablet_copy_client.cc:523] T 5f84e791335948e180f5575f8174efd1 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:21.435024 12158 tablet_bootstrap.cc:492] T 5f84e791335948e180f5575f8174efd1 P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap starting.
I20240502 14:01:21.456905 12153 tablet_bootstrap.cc:492] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:21.457298 12153 tablet_bootstrap.cc:492] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap complete.
I20240502 14:01:21.457497 12153 ts_tablet_manager.cc:1367] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent bootstrapping tablet: real 0.035s user 0.029s sys 0.000s
I20240502 14:01:21.457979 12153 raft_consensus.cc:348] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:21.458123 12153 raft_consensus.cc:729] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e01cf117ef5d446590c06dd1a6ab7ef7, State: Initialized, Role: LEARNER
I20240502 14:01:21.458225 12153 consensus_queue.cc:260] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:21.458629 12153 ts_tablet_manager.cc:1398] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:21.461535 9190 tablet_copy_service.cc:342] P d83af1bc823a4f2ca531f0d6825f8e16: Request end of tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-29eec0daa5d34e40a4ad82607b7ded90 received from {username='slave'} at 127.24.66.72:37077
I20240502 14:01:21.461663 9190 tablet_copy_service.cc:433] P d83af1bc823a4f2ca531f0d6825f8e16: ending tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-29eec0daa5d34e40a4ad82607b7ded90 on tablet 29eec0daa5d34e40a4ad82607b7ded90 with peer e01cf117ef5d446590c06dd1a6ab7ef7
I20240502 14:01:21.467095 12158 tablet_bootstrap.cc:492] T 5f84e791335948e180f5575f8174efd1 P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:21.467409 12158 tablet_bootstrap.cc:492] T 5f84e791335948e180f5575f8174efd1 P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap complete.
I20240502 14:01:21.467602 12158 ts_tablet_manager.cc:1367] T 5f84e791335948e180f5575f8174efd1 P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent bootstrapping tablet: real 0.033s user 0.027s sys 0.000s
I20240502 14:01:21.468014 12158 raft_consensus.cc:348] T 5f84e791335948e180f5575f8174efd1 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:21.468137 12158 raft_consensus.cc:729] T 5f84e791335948e180f5575f8174efd1 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 535b19a469c34efaaf49ca1cc2e1e5fc, State: Initialized, Role: LEARNER
I20240502 14:01:21.468494 12158 consensus_queue.cc:260] T 5f84e791335948e180f5575f8174efd1 P 535b19a469c34efaaf49ca1cc2e1e5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:21.468849 12158 ts_tablet_manager.cc:1398] T 5f84e791335948e180f5575f8174efd1 P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 14:01:21.469501 9454 tablet_copy_service.cc:342] P 1a490f170c614ee4900ef16c3287c7de: Request end of tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-5f84e791335948e180f5575f8174efd1 received from {username='slave'} at 127.24.66.74:50277
I20240502 14:01:21.469609 9454 tablet_copy_service.cc:433] P 1a490f170c614ee4900ef16c3287c7de: ending tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-5f84e791335948e180f5575f8174efd1 on tablet 5f84e791335948e180f5575f8174efd1 with peer 535b19a469c34efaaf49ca1cc2e1e5fc
I20240502 14:01:21.485582 11231 raft_consensus.cc:1203] T b1548ca1bcd24d288d30117baea53c12 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 14:01:21.581985 12153 ts_tablet_manager.cc:902] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7: Initiating tablet copy from peer d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837)
I20240502 14:01:21.582676 12153 tablet_copy_client.cc:308] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.66:38837
I20240502 14:01:21.583262 9190 tablet_copy_service.cc:140] P d83af1bc823a4f2ca531f0d6825f8e16: Received BeginTabletCopySession request for tablet ada4c2cc1749411d8f41f882b7c64b07 from peer e01cf117ef5d446590c06dd1a6ab7ef7 ({username='slave'} at 127.24.66.72:37077)
I20240502 14:01:21.583377 9190 tablet_copy_service.cc:161] P d83af1bc823a4f2ca531f0d6825f8e16: Beginning new tablet copy session on tablet ada4c2cc1749411d8f41f882b7c64b07 from peer e01cf117ef5d446590c06dd1a6ab7ef7 at {username='slave'} at 127.24.66.72:37077: session id = e01cf117ef5d446590c06dd1a6ab7ef7-ada4c2cc1749411d8f41f882b7c64b07
I20240502 14:01:21.584139 9190 tablet_copy_source_session.cc:205] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:21.585114 12153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ada4c2cc1749411d8f41f882b7c64b07. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:21.587060 12153 tablet_copy_client.cc:783] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:21.587237 12153 tablet_copy_client.cc:647] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:21.588431 12153 tablet_copy_client.cc:523] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:21.589612 12153 tablet_bootstrap.cc:492] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap starting.
I20240502 14:01:21.610190 12153 tablet_bootstrap.cc:492] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:21.610479 12153 tablet_bootstrap.cc:492] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap complete.
I20240502 14:01:21.610674 12153 ts_tablet_manager.cc:1367] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent bootstrapping tablet: real 0.021s user 0.020s sys 0.000s
I20240502 14:01:21.611088 12153 raft_consensus.cc:348] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:21.611227 12153 raft_consensus.cc:729] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e01cf117ef5d446590c06dd1a6ab7ef7, State: Initialized, Role: LEARNER
I20240502 14:01:21.611342 12153 consensus_queue.cc:260] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:21.611945 12153 ts_tablet_manager.cc:1398] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 14:01:21.612445 9190 tablet_copy_service.cc:342] P d83af1bc823a4f2ca531f0d6825f8e16: Request end of tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-ada4c2cc1749411d8f41f882b7c64b07 received from {username='slave'} at 127.24.66.72:37077
I20240502 14:01:21.612547 9190 tablet_copy_service.cc:433] P d83af1bc823a4f2ca531f0d6825f8e16: ending tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-ada4c2cc1749411d8f41f882b7c64b07 on tablet ada4c2cc1749411d8f41f882b7c64b07 with peer e01cf117ef5d446590c06dd1a6ab7ef7
I20240502 14:01:21.809537 12158 ts_tablet_manager.cc:902] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc: Initiating tablet copy from peer 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:21.810117 12158 tablet_copy_client.cc:308] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.71:33679
I20240502 14:01:21.814620 9850 tablet_copy_service.cc:140] P 0488b0e4ed994db19d40aed8c15f945b: Received BeginTabletCopySession request for tablet 655c86a7f52a40c68c3e95cb12286c29 from peer 535b19a469c34efaaf49ca1cc2e1e5fc ({username='slave'} at 127.24.66.74:53801)
I20240502 14:01:21.814759 9850 tablet_copy_service.cc:161] P 0488b0e4ed994db19d40aed8c15f945b: Beginning new tablet copy session on tablet 655c86a7f52a40c68c3e95cb12286c29 from peer 535b19a469c34efaaf49ca1cc2e1e5fc at {username='slave'} at 127.24.66.74:53801: session id = 535b19a469c34efaaf49ca1cc2e1e5fc-655c86a7f52a40c68c3e95cb12286c29
I20240502 14:01:21.816099 9850 tablet_copy_source_session.cc:205] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:21.817420 12158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 655c86a7f52a40c68c3e95cb12286c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:21.819478 12158 tablet_copy_client.cc:783] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:21.819630 12158 tablet_copy_client.cc:647] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:21.820960 12158 tablet_copy_client.cc:523] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:21.822106 12158 tablet_bootstrap.cc:492] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap starting.
I20240502 14:01:21.843506 12158 tablet_bootstrap.cc:492] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:21.843839 12158 tablet_bootstrap.cc:492] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap complete.
I20240502 14:01:21.844050 12158 ts_tablet_manager.cc:1367] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent bootstrapping tablet: real 0.022s user 0.019s sys 0.000s
I20240502 14:01:21.844556 12158 raft_consensus.cc:348] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:21.844687 12158 raft_consensus.cc:729] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 535b19a469c34efaaf49ca1cc2e1e5fc, State: Initialized, Role: LEARNER
I20240502 14:01:21.844810 12158 consensus_queue.cc:260] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:21.845108 12158 ts_tablet_manager.cc:1398] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 14:01:21.845679 9850 tablet_copy_service.cc:342] P 0488b0e4ed994db19d40aed8c15f945b: Request end of tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-655c86a7f52a40c68c3e95cb12286c29 received from {username='slave'} at 127.24.66.74:53801
I20240502 14:01:21.845789 9850 tablet_copy_service.cc:433] P 0488b0e4ed994db19d40aed8c15f945b: ending tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-655c86a7f52a40c68c3e95cb12286c29 on tablet 655c86a7f52a40c68c3e95cb12286c29 with peer 535b19a469c34efaaf49ca1cc2e1e5fc
I20240502 14:01:21.859746 11231 raft_consensus.cc:1203] T 50451fdbd43b491fa701daac5de5e1c8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 14:01:21.890703 11231 raft_consensus.cc:1203] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 14:01:21.935287 11362 raft_consensus.cc:1203] T 5f84e791335948e180f5575f8174efd1 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 14:01:21.958926 11858 raft_consensus.cc:1050] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de: attempting to promote NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 to VOTER
I20240502 14:01:21.959494 11858 consensus_queue.cc:237] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } }
I20240502 14:01:21.961555 9828 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:21.961712 9564 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:21.961935 9301 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:21.962011 9170 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:21.962209 11231 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:21.962368 12249 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:21.962445 9697 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:21.962457 9036 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:21.962733 12251 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:21.963085 11860 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:21.963234 11945 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:21.964490 11860 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:21.964762 11945 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:21.964879 11860 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:21.967511 12251 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:21.967963 9301 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:21.969075 9697 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:21.969981 9564 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:21.971388 11231 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:21.972716 8896 catalog_manager.cc:5484] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:21.974081 9828 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:21.974382 9036 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:21.977279 9170 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:21.982937 9434 consensus_queue.cc:237] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } }
I20240502 14:01:21.985378 9697 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:21.985682 11231 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:21.986073 9301 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:21.986207 11858 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:21.986541 9828 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:21.986598 11858 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:21.986716 9036 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:21.987169 12250 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:21.988538 11858 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:21.988759 11860 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:21.989351 9170 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:21.990221 11860 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:21.991014 9301 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:21.993863 11231 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:21.994060 12248 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:21.994210 8889 catalog_manager.cc:5484] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:21.995064 9830 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:21.995200 9036 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:21.996521 8882 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet b1548ca1bcd24d288d30117baea53c12 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:21.997467 9697 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:21.998381 9170 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.006657 9544 tablet_service.cc:1514] Processing DeleteTablet for tablet b1548ca1bcd24d288d30117baea53c12 with delete_type TABLET_DATA_TOMBSTONED (TS d0f45862d7f14ce1976c1a37842f25ca not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:46242
I20240502 14:01:22.007000 12263 tablet_replica.cc:330] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca: stopping tablet replica
I20240502 14:01:22.007129 12263 raft_consensus.cc:2229] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:22.007325 12263 raft_consensus.cc:2258] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:22.008862 12263 ts_tablet_manager.cc:1875] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:22.011641 12263 ts_tablet_manager.cc:1888] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 14:01:22.011756 12263 log.cc:1196] T b1548ca1bcd24d288d30117baea53c12 P d0f45862d7f14ce1976c1a37842f25ca: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal/wals/b1548ca1bcd24d288d30117baea53c12
I20240502 14:01:22.012292 8881 catalog_manager.cc:4830] TS d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375): tablet b1548ca1bcd24d288d30117baea53c12 (table rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]) successfully deleted
I20240502 14:01:22.125789 11231 raft_consensus.cc:1203] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 14:01:22.264549 12048 raft_consensus.cc:1050] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f: attempting to promote NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 to VOTER
I20240502 14:01:22.265044 12048 consensus_queue.cc:237] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } }
I20240502 14:01:22.267174 11231 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.267640 12046 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.267782 9170 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.267865 9036 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.268118 12046 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.268986 12046 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.270061 9564 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.270620 9434 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.271456 12047 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.271538 9697 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.271912 12047 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.272114 12046 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.272977 9828 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.274008 12046 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.276275 12048 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.276786 9828 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.278879 11231 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.279410 8890 catalog_manager.cc:5484] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:22.280812 9434 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.280969 9170 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.284150 9036 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.286280 9564 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.290733 9697 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.292165 9301 consensus_queue.cc:237] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } }
I20240502 14:01:22.293634 11231 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.294186 9564 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.294186 9036 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.294245 9170 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.294660 9697 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.294646 12046 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.294749 12049 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.294906 12047 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.295161 12046 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.295282 9828 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.295403 12048 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.296737 12048 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.297315 12046 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.298650 8880 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 50451fdbd43b491fa701daac5de5e1c8 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:22.298557 9564 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.299057 9168 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.299274 11231 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.301950 8890 catalog_manager.cc:5484] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:22.302742 9828 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.302762 9036 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.307092 9697 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.308581 9412 tablet_service.cc:1514] Processing DeleteTablet for tablet 50451fdbd43b491fa701daac5de5e1c8 with delete_type TABLET_DATA_TOMBSTONED (TS 1a490f170c614ee4900ef16c3287c7de not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:33738
I20240502 14:01:22.311210 12303 tablet_replica.cc:330] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de: stopping tablet replica
I20240502 14:01:22.311379 12303 raft_consensus.cc:2229] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:22.311601 12303 raft_consensus.cc:2258] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:22.312279 12303 ts_tablet_manager.cc:1875] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:22.314287 12303 ts_tablet_manager.cc:1888] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 14:01:22.314378 12303 log.cc:1196] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/50451fdbd43b491fa701daac5de5e1c8
I20240502 14:01:22.314579 11362 raft_consensus.cc:1203] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 14:01:22.314924 8882 catalog_manager.cc:4830] TS 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907): tablet 50451fdbd43b491fa701daac5de5e1c8 (table rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]) successfully deleted
I20240502 14:01:22.317616 11860 raft_consensus.cc:1050] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de: attempting to promote NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc to VOTER
I20240502 14:01:22.318096 11860 consensus_queue.cc:237] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:22.320047 9168 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.320047 9301 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.320447 11858 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.320547 9828 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.320684 11362 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.321094 11858 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.321152 9697 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.321374 12250 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.322158 9564 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.323031 12250 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.323300 12248 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.323472 11858 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.324204 9036 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.325827 11982 raft_consensus.cc:1050] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16: attempting to promote NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 to VOTER
I20240502 14:01:22.326200 12248 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.326368 11982 consensus_queue.cc:237] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } }
I20240502 14:01:22.326465 12250 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.326949 9828 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.328306 11231 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.328810 11362 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.329607 9566 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.329675 9564 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.329977 11990 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.330358 9036 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.331641 9038 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.332038 12313 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.332106 9302 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.332372 9696 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.332686 9828 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.333004 9434 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.333452 8895 catalog_manager.cc:5484] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d reported cstate change: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.334390 11659 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.334501 9168 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.334815 11659 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.335117 11659 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.336282 9697 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.338723 11982 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.339280 11990 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.342417 11990 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.344547 9566 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.344897 11231 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.345842 9301 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.347236 9302 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.348304 8890 catalog_manager.cc:5484] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7 reported cstate change: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.348616 9038 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.349777 9434 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.354346 9697 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.355067 9828 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.358584 9168 consensus_queue.cc:237] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } }
I20240502 14:01:22.360081 9828 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.360535 11231 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.360757 9302 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.361047 9697 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.361155 12314 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.362286 9434 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.361586 12314 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.362807 9038 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.362828 11990 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.363265 11990 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.363580 11990 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.363878 11659 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.366767 11659 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.368196 11231 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.369199 8882 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 29eec0daa5d34e40a4ad82607b7ded90 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:22.369730 9828 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.369730 9302 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.371651 8896 catalog_manager.cc:5484] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7 reported cstate change: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.372135 9038 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.373116 9697 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.375784 9434 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.379413 9544 tablet_service.cc:1514] Processing DeleteTablet for tablet 29eec0daa5d34e40a4ad82607b7ded90 with delete_type TABLET_DATA_TOMBSTONED (TS d0f45862d7f14ce1976c1a37842f25ca not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:46242
I20240502 14:01:22.379714 12263 tablet_replica.cc:330] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca: stopping tablet replica
I20240502 14:01:22.379874 12263 raft_consensus.cc:2229] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:22.380096 12263 raft_consensus.cc:2258] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:22.381421 12263 ts_tablet_manager.cc:1875] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:22.381968 9434 consensus_queue.cc:237] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:22.383783 9168 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:22.383908 12263 ts_tablet_manager.cc:1888] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 14:01:22.383947 9302 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:22.383984 12263 log.cc:1196] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal/wals/29eec0daa5d34e40a4ad82607b7ded90
I20240502 14:01:22.384488 11860 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:22.384578 8881 catalog_manager.cc:4830] TS d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375): tablet 29eec0daa5d34e40a4ad82607b7ded90 (table rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]) successfully deleted
I20240502 14:01:22.384902 9697 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:22.385260 9828 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:22.385682 12250 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:22.385872 9566 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:22.385988 12250 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:22.386283 12248 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:22.387029 9038 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:22.387444 11860 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:22.388041 11860 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:22.389446 12248 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:22.390573 9302 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:22.391453 9697 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:22.391984 9828 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:22.393677 8896 catalog_manager.cc:5484] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:22.394191 9566 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:22.396101 9168 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:22.397258 9038 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:22.402913 9434 consensus_queue.cc:237] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:22.406342 9302 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:22.406380 9697 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:22.406385 9828 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240502 14:01:22.406508 9370 consensus_peers.cc:485] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de -> Peer 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74:34421): Couldn't send request to peer 535b19a469c34efaaf49ca1cc2e1e5fc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6aed24ef77e54c8b8ff09e6709a886e4. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:22.406692 11858 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:22.406816 9566 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:22.406903 9038 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:22.407364 11945 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:22.407418 9168 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:22.407496 11858 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:22.407776 11858 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:22.407922 12248 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:22.408039 12251 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:22.410494 11858 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:22.410895 9697 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:22.413259 8882 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 6aed24ef77e54c8b8ff09e6709a886e4 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 14:01:22.414083 9302 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:22.414763 8892 catalog_manager.cc:5484] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de reported cstate change: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:22.414878 9566 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:22.418618 9038 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:22.419943 9828 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:22.425307 9170 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:22.633076 12314 raft_consensus.cc:1050] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16: attempting to promote NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 to VOTER
I20240502 14:01:22.633605 12314 consensus_queue.cc:237] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } }
I20240502 14:01:22.635485 9566 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.635588 9828 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.635735 9697 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.636050 11231 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.636050 9433 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.636236 12313 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.636490 12369 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.636657 11990 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.636814 11982 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.637151 9302 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.637986 11990 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.638320 11990 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.639916 9038 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.640388 11990 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.641330 12313 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.642309 9302 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.642818 9566 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.644556 9696 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.645982 11231 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.646847 9828 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.647990 8896 catalog_manager.cc:5484] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.648175 9433 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.652531 9036 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.657655 9170 consensus_queue.cc:237] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } }
I20240502 14:01:22.659267 9038 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.659256 9566 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.659718 12369 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.659808 9828 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.659879 11231 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.660193 9696 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.660445 12314 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.660573 9302 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.660799 12314 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.661509 12324 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.661736 11659 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.661864 11982 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.664644 12324 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.664938 11231 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.666102 9696 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.667884 8892 catalog_manager.cc:5484] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7 reported cstate change: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.668417 9566 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.670392 8882 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ada4c2cc1749411d8f41f882b7c64b07 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:22.670768 9412 tablet_service.cc:1514] Processing DeleteTablet for tablet ada4c2cc1749411d8f41f882b7c64b07 with delete_type TABLET_DATA_TOMBSTONED (TS 1a490f170c614ee4900ef16c3287c7de not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:33738
I20240502 14:01:22.671106 12303 tablet_replica.cc:330] T ada4c2cc1749411d8f41f882b7c64b07 P 1a490f170c614ee4900ef16c3287c7de: stopping tablet replica
I20240502 14:01:22.670991 9828 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.671267 12303 raft_consensus.cc:2229] T ada4c2cc1749411d8f41f882b7c64b07 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:22.671481 12303 raft_consensus.cc:2258] T ada4c2cc1749411d8f41f882b7c64b07 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:22.671689 9038 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.672173 9302 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:22.674965 12303 ts_tablet_manager.cc:1875] T ada4c2cc1749411d8f41f882b7c64b07 P 1a490f170c614ee4900ef16c3287c7de: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:22.677204 12303 ts_tablet_manager.cc:1888] T ada4c2cc1749411d8f41f882b7c64b07 P 1a490f170c614ee4900ef16c3287c7de: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 14:01:22.677297 12303 log.cc:1196] T ada4c2cc1749411d8f41f882b7c64b07 P 1a490f170c614ee4900ef16c3287c7de: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/ada4c2cc1749411d8f41f882b7c64b07
I20240502 14:01:22.677846 8882 catalog_manager.cc:4830] TS 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907): tablet ada4c2cc1749411d8f41f882b7c64b07 (table rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]) successfully deleted
I20240502 14:01:22.679279 9302 consensus_queue.cc:237] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:22.681277 9696 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:22.681547 9828 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:22.681661 9566 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:22.681983 9433 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:22.682113 12047 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:22.682214 9038 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:22.682504 12047 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:22.682528 9170 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:22.682646 12048 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:22.682881 12301 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:22.683126 12048 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:22.683650 12048 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:22.686281 12047 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:22.686653 9433 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:22.686894 9038 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:22.688373 9170 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:22.689437 9696 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:22.690968 8888 catalog_manager.cc:5484] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:22.693354 9566 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:22.693310 9828 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:22.698850 9302 consensus_queue.cc:237] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:22.701766 9566 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:22.701774 9696 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:22.702168 12048 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:22.702138 9828 consensus_queue.cc:237] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:22.702360 12047 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:22.703203 9170 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:22.703572 9433 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:22.703946 9830 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:22.704006 9038 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:22.704066 12047 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:22.704463 12047 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:22.704607 12049 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:22.704921 12046 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:22.704957 9566 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:22.705322 12107 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:22.705427 9168 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:22.706104 9038 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:22.706485 12304 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:22.706640 9433 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:22.706800 12304 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:22.707475 12129 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:22.707742 9302 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:22.708156 9696 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:22.708839 12129 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:22.709151 12129 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:22.709211 12046 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
W20240502 14:01:22.711006 9238 consensus_peers.cc:485] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f -> Peer 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74:34421): Couldn't send request to peer 535b19a469c34efaaf49ca1cc2e1e5fc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fed16a08a40a474a9159826ca8731acb. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:22.713282 9564 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:22.713313 9830 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:22.713539 9036 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:22.714717 9433 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:22.712971 8888 catalog_manager.cc:5484] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:22.715498 9697 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:22.715829 8880 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet fed16a08a40a474a9159826ca8731acb with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 14:01:22.718693 12107 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:22.719102 9697 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:22.719452 9170 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:22.720808 9566 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:22.721999 9302 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:22.725593 9434 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:22.726204 9168 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:22.726315 9038 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:22.731114 8895 catalog_manager.cc:5484] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:22.738445 9830 consensus_queue.cc:237] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:22.740826 9566 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:22.741380 12304 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:22.741458 9302 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:22.741868 12304 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:22.742501 9168 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:22.742648 9434 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:22.742748 9038 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:22.742884 9697 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240502 14:01:22.742954 9765 consensus_peers.cc:485] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b -> Peer 304274b88d234198a3892e38c05014a5 (127.24.66.73:42559): Couldn't send request to peer 304274b88d234198a3892e38c05014a5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6232f47837664d7486612ba1d291f2b8. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:22.743115 12129 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:22.743391 12129 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:22.743700 12129 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:22.743988 12107 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:22.745481 12107 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:22.748258 9566 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:22.748234 9301 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:22.748476 8880 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 6232f47837664d7486612ba1d291f2b8 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 14:01:22.748842 9168 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:22.749400 8893 catalog_manager.cc:5484] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:22.750324 9038 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:22.750795 9434 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:22.752636 9697 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:22.834066 12412 raft_consensus.cc:1050] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b: attempting to promote NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc to VOTER
I20240502 14:01:22.834659 12412 consensus_queue.cc:237] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:22.836524 9434 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.836858 9566 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.836933 11362 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.837127 9301 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.837169 12107 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.837536 12304 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.837703 9038 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.837791 9697 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.837890 12129 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.838083 12107 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.838295 9168 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:22.839303 12412 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.840315 12304 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.840600 12129 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:22.842566 12412 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.843230 9697 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.845045 9564 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.845415 9168 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.845929 8889 catalog_manager.cc:5484] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:22.846683 11362 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.848001 9038 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.850023 9301 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.850543 9433 consensus_queue.cc:237] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:22.853179 11362 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.853572 12248 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.854089 9830 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.854089 9038 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.854883 12248 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.855170 12249 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.855669 9697 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.856056 12249 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.858489 9301 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.858542 12248 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.859244 12250 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.859158 9434 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.859388 11362 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.860121 9830 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.861429 8882 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5f84e791335948e180f5575f8174efd1 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:22.861953 9697 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.862031 9038 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.862685 8896 catalog_manager.cc:5484] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de reported cstate change: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:22.863534 9168 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.864116 12249 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.865290 9301 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.866566 9168 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.868113 9544 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f84e791335948e180f5575f8174efd1 with delete_type TABLET_DATA_TOMBSTONED (TS d0f45862d7f14ce1976c1a37842f25ca not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:46242
I20240502 14:01:22.868481 12263 tablet_replica.cc:330] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca: stopping tablet replica
I20240502 14:01:22.868642 12263 raft_consensus.cc:2229] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:22.868810 12263 raft_consensus.cc:2258] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:22.869683 12263 ts_tablet_manager.cc:1875] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:22.871755 12263 ts_tablet_manager.cc:1888] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 14:01:22.871819 12263 log.cc:1196] T 5f84e791335948e180f5575f8174efd1 P d0f45862d7f14ce1976c1a37842f25ca: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal/wals/5f84e791335948e180f5575f8174efd1
I20240502 14:01:22.872366 8881 catalog_manager.cc:4830] TS d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375): tablet 5f84e791335948e180f5575f8174efd1 (table rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]) successfully deleted
I20240502 14:01:22.875310 9830 consensus_queue.cc:237] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:22.878928 9564 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.879081 11362 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.879335 12304 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.879540 9170 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.879647 9697 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.880005 9301 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.880110 12414 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.880385 12412 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.880519 12414 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.880726 12304 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.881385 9038 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:22.881806 12304 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:22.883147 12412 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.883677 11362 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.884446 9302 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.885049 9564 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.887022 9168 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.887591 8889 catalog_manager.cc:5484] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca reported cstate change: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.888815 9697 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.890622 9412 tablet_service.cc:1514] Processing DeleteTablet for tablet 655c86a7f52a40c68c3e95cb12286c29 with delete_type TABLET_DATA_TOMBSTONED (TS 1a490f170c614ee4900ef16c3287c7de not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:33738
I20240502 14:01:22.890944 12303 tablet_replica.cc:330] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de: stopping tablet replica
I20240502 14:01:22.890928 8880 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 655c86a7f52a40c68c3e95cb12286c29 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:22.891088 12303 raft_consensus.cc:2229] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:22.891278 12303 raft_consensus.cc:2258] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:22.891546 9038 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:22.892242 12303 ts_tablet_manager.cc:1875] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:22.894707 12303 ts_tablet_manager.cc:1888] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 14:01:22.894790 12303 log.cc:1196] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/655c86a7f52a40c68c3e95cb12286c29
I20240502 14:01:22.895376 8882 catalog_manager.cc:4830] TS 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907): tablet 655c86a7f52a40c68c3e95cb12286c29 (table rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]) successfully deleted
I20240502 14:01:22.919374 12446 ts_tablet_manager.cc:902] T 6aed24ef77e54c8b8ff09e6709a886e4 P 535b19a469c34efaaf49ca1cc2e1e5fc: Initiating tablet copy from peer 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907)
I20240502 14:01:22.920006 12446 tablet_copy_client.cc:308] T 6aed24ef77e54c8b8ff09e6709a886e4 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:42907
I20240502 14:01:22.920572 9454 tablet_copy_service.cc:140] P 1a490f170c614ee4900ef16c3287c7de: Received BeginTabletCopySession request for tablet 6aed24ef77e54c8b8ff09e6709a886e4 from peer 535b19a469c34efaaf49ca1cc2e1e5fc ({username='slave'} at 127.24.66.74:50277)
I20240502 14:01:22.920706 9454 tablet_copy_service.cc:161] P 1a490f170c614ee4900ef16c3287c7de: Beginning new tablet copy session on tablet 6aed24ef77e54c8b8ff09e6709a886e4 from peer 535b19a469c34efaaf49ca1cc2e1e5fc at {username='slave'} at 127.24.66.74:50277: session id = 535b19a469c34efaaf49ca1cc2e1e5fc-6aed24ef77e54c8b8ff09e6709a886e4
I20240502 14:01:22.921567 9454 tablet_copy_source_session.cc:205] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:22.922478 12446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aed24ef77e54c8b8ff09e6709a886e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:22.924363 12446 tablet_copy_client.cc:783] T 6aed24ef77e54c8b8ff09e6709a886e4 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:22.924623 12446 tablet_copy_client.cc:647] T 6aed24ef77e54c8b8ff09e6709a886e4 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:22.925776 12446 tablet_copy_client.cc:523] T 6aed24ef77e54c8b8ff09e6709a886e4 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:22.926829 12446 tablet_bootstrap.cc:492] T 6aed24ef77e54c8b8ff09e6709a886e4 P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap starting.
I20240502 14:01:22.945950 12446 tablet_bootstrap.cc:492] T 6aed24ef77e54c8b8ff09e6709a886e4 P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:22.946295 12446 tablet_bootstrap.cc:492] T 6aed24ef77e54c8b8ff09e6709a886e4 P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap complete.
I20240502 14:01:22.946496 12446 ts_tablet_manager.cc:1367] T 6aed24ef77e54c8b8ff09e6709a886e4 P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent bootstrapping tablet: real 0.020s user 0.017s sys 0.001s
I20240502 14:01:22.946930 12446 raft_consensus.cc:348] T 6aed24ef77e54c8b8ff09e6709a886e4 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:22.947052 12446 raft_consensus.cc:729] T 6aed24ef77e54c8b8ff09e6709a886e4 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 535b19a469c34efaaf49ca1cc2e1e5fc, State: Initialized, Role: LEARNER
I20240502 14:01:22.947156 12446 consensus_queue.cc:260] T 6aed24ef77e54c8b8ff09e6709a886e4 P 535b19a469c34efaaf49ca1cc2e1e5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:22.947444 12446 ts_tablet_manager.cc:1398] T 6aed24ef77e54c8b8ff09e6709a886e4 P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:22.948041 9454 tablet_copy_service.cc:342] P 1a490f170c614ee4900ef16c3287c7de: Request end of tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-6aed24ef77e54c8b8ff09e6709a886e4 received from {username='slave'} at 127.24.66.74:50277
I20240502 14:01:22.948137 9454 tablet_copy_service.cc:433] P 1a490f170c614ee4900ef16c3287c7de: ending tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-6aed24ef77e54c8b8ff09e6709a886e4 on tablet 6aed24ef77e54c8b8ff09e6709a886e4 with peer 535b19a469c34efaaf49ca1cc2e1e5fc
I20240502 14:01:22.991591 9564 consensus_queue.cc:237] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:22.993993 9830 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:22.994191 9168 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:22.994429 9038 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:22.994693 9434 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:22.994733 9697 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:22.994777 9302 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:22.995160 12461 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:22.995391 11950 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:22.995546 12462 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:22.995869 11950 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:22.996018 12461 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:22.997831 12463 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:22.999378 11950 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:22.999831 9302 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:23.002640 8893 catalog_manager.cc:5484] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:23.004559 9434 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:23.004746 9038 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:23.010497 9830 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:23.010840 9564 consensus_queue.cc:237] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:23.013760 9434 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:23.013684 9697 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:23.014501 9168 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:23.014680 9830 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:23.014905 12460 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:23.015110 11950 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:23.015209 12462 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:23.016939 9038 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:23.017627 9302 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:23.017827 9168 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:23.018110 12474 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:23.019840 11950 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:23.020933 9302 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:23.022552 8881 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 21d4acfb937f4dd4b992c338fc6b3237 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 14:01:23.022570 12459 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:23.023414 9830 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:23.024253 9038 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:23.024397 8892 catalog_manager.cc:5484] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca reported cstate change: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:23.025077 9168 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:23.026225 9434 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:23.029590 9697 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
W20240502 14:01:23.034909 9502 consensus_peers.cc:485] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca -> Peer 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74:34421): Couldn't send request to peer 535b19a469c34efaaf49ca1cc2e1e5fc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 21d4acfb937f4dd4b992c338fc6b3237. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:23.155500 12446 ts_tablet_manager.cc:902] T fed16a08a40a474a9159826ca8731acb P 535b19a469c34efaaf49ca1cc2e1e5fc: Initiating tablet copy from peer ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:23.156044 12446 tablet_copy_client.cc:308] T fed16a08a40a474a9159826ca8731acb P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.67:34749
I20240502 14:01:23.159686 9322 tablet_copy_service.cc:140] P ccfb5bbc1bb54793812d11170ac2d25f: Received BeginTabletCopySession request for tablet fed16a08a40a474a9159826ca8731acb from peer 535b19a469c34efaaf49ca1cc2e1e5fc ({username='slave'} at 127.24.66.74:58125)
I20240502 14:01:23.159812 9322 tablet_copy_service.cc:161] P ccfb5bbc1bb54793812d11170ac2d25f: Beginning new tablet copy session on tablet fed16a08a40a474a9159826ca8731acb from peer 535b19a469c34efaaf49ca1cc2e1e5fc at {username='slave'} at 127.24.66.74:58125: session id = 535b19a469c34efaaf49ca1cc2e1e5fc-fed16a08a40a474a9159826ca8731acb
I20240502 14:01:23.160806 9322 tablet_copy_source_session.cc:205] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:23.161834 12446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fed16a08a40a474a9159826ca8731acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:23.163887 12446 tablet_copy_client.cc:783] T fed16a08a40a474a9159826ca8731acb P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:23.164043 12446 tablet_copy_client.cc:647] T fed16a08a40a474a9159826ca8731acb P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:23.165298 12446 tablet_copy_client.cc:523] T fed16a08a40a474a9159826ca8731acb P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:23.166618 12446 tablet_bootstrap.cc:492] T fed16a08a40a474a9159826ca8731acb P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap starting.
I20240502 14:01:23.183970 12509 ts_tablet_manager.cc:902] T 6232f47837664d7486612ba1d291f2b8 P 304274b88d234198a3892e38c05014a5: Initiating tablet copy from peer 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:23.184582 12509 tablet_copy_client.cc:308] T 6232f47837664d7486612ba1d291f2b8 P 304274b88d234198a3892e38c05014a5: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.71:33679
I20240502 14:01:23.185122 9850 tablet_copy_service.cc:140] P 0488b0e4ed994db19d40aed8c15f945b: Received BeginTabletCopySession request for tablet 6232f47837664d7486612ba1d291f2b8 from peer 304274b88d234198a3892e38c05014a5 ({username='slave'} at 127.24.66.73:58115)
I20240502 14:01:23.185245 9850 tablet_copy_service.cc:161] P 0488b0e4ed994db19d40aed8c15f945b: Beginning new tablet copy session on tablet 6232f47837664d7486612ba1d291f2b8 from peer 304274b88d234198a3892e38c05014a5 at {username='slave'} at 127.24.66.73:58115: session id = 304274b88d234198a3892e38c05014a5-6232f47837664d7486612ba1d291f2b8
I20240502 14:01:23.186156 9850 tablet_copy_source_session.cc:205] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:23.186482 12446 tablet_bootstrap.cc:492] T fed16a08a40a474a9159826ca8731acb P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:23.186797 12446 tablet_bootstrap.cc:492] T fed16a08a40a474a9159826ca8731acb P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap complete.
I20240502 14:01:23.187016 12446 ts_tablet_manager.cc:1367] T fed16a08a40a474a9159826ca8731acb P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent bootstrapping tablet: real 0.020s user 0.019s sys 0.000s
I20240502 14:01:23.187141 12509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6232f47837664d7486612ba1d291f2b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:23.187577 12446 raft_consensus.cc:348] T fed16a08a40a474a9159826ca8731acb P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:23.187744 12446 raft_consensus.cc:729] T fed16a08a40a474a9159826ca8731acb P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 535b19a469c34efaaf49ca1cc2e1e5fc, State: Initialized, Role: LEARNER
I20240502 14:01:23.187875 12446 consensus_queue.cc:260] T fed16a08a40a474a9159826ca8731acb P 535b19a469c34efaaf49ca1cc2e1e5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:23.188215 12446 ts_tablet_manager.cc:1398] T fed16a08a40a474a9159826ca8731acb P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:23.188815 9322 tablet_copy_service.cc:342] P ccfb5bbc1bb54793812d11170ac2d25f: Request end of tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-fed16a08a40a474a9159826ca8731acb received from {username='slave'} at 127.24.66.74:58125
I20240502 14:01:23.188916 9322 tablet_copy_service.cc:433] P ccfb5bbc1bb54793812d11170ac2d25f: ending tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-fed16a08a40a474a9159826ca8731acb on tablet fed16a08a40a474a9159826ca8731acb with peer 535b19a469c34efaaf49ca1cc2e1e5fc
I20240502 14:01:23.189586 12509 tablet_copy_client.cc:783] T 6232f47837664d7486612ba1d291f2b8 P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:23.189755 12509 tablet_copy_client.cc:647] T 6232f47837664d7486612ba1d291f2b8 P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:23.191242 12509 tablet_copy_client.cc:523] T 6232f47837664d7486612ba1d291f2b8 P 304274b88d234198a3892e38c05014a5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:23.192478 12509 tablet_bootstrap.cc:492] T 6232f47837664d7486612ba1d291f2b8 P 304274b88d234198a3892e38c05014a5: Bootstrap starting.
I20240502 14:01:23.214229 12509 tablet_bootstrap.cc:492] T 6232f47837664d7486612ba1d291f2b8 P 304274b88d234198a3892e38c05014a5: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:23.214613 12509 tablet_bootstrap.cc:492] T 6232f47837664d7486612ba1d291f2b8 P 304274b88d234198a3892e38c05014a5: Bootstrap complete.
I20240502 14:01:23.214825 12509 ts_tablet_manager.cc:1367] T 6232f47837664d7486612ba1d291f2b8 P 304274b88d234198a3892e38c05014a5: Time spent bootstrapping tablet: real 0.022s user 0.020s sys 0.000s
I20240502 14:01:23.215276 12509 raft_consensus.cc:348] T 6232f47837664d7486612ba1d291f2b8 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:23.215425 12509 raft_consensus.cc:729] T 6232f47837664d7486612ba1d291f2b8 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 304274b88d234198a3892e38c05014a5, State: Initialized, Role: LEARNER
I20240502 14:01:23.215555 12509 consensus_queue.cc:260] T 6232f47837664d7486612ba1d291f2b8 P 304274b88d234198a3892e38c05014a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:23.215975 12509 ts_tablet_manager.cc:1398] T 6232f47837664d7486612ba1d291f2b8 P 304274b88d234198a3892e38c05014a5: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 14:01:23.216499 9850 tablet_copy_service.cc:342] P 0488b0e4ed994db19d40aed8c15f945b: Request end of tablet copy session 304274b88d234198a3892e38c05014a5-6232f47837664d7486612ba1d291f2b8 received from {username='slave'} at 127.24.66.73:58115
I20240502 14:01:23.216583 9850 tablet_copy_service.cc:433] P 0488b0e4ed994db19d40aed8c15f945b: ending tablet copy session 304274b88d234198a3892e38c05014a5-6232f47837664d7486612ba1d291f2b8 on tablet 6232f47837664d7486612ba1d291f2b8 with peer 304274b88d234198a3892e38c05014a5
I20240502 14:01:23.289009 9564 consensus_queue.cc:237] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:23.290871 9302 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:23.291114 9697 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:23.291208 9168 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:23.291239 9830 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:23.291462 12462 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:23.291777 12461 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:23.291958 12475 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:23.292235 12462 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:23.292660 9038 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:23.293179 12462 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:23.294607 9434 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:23.295264 12462 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:23.297252 12462 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:23.299453 9434 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:23.299664 9302 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:23.301924 9168 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:23.302234 9830 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:23.302368 8893 catalog_manager.cc:5484] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:23.304653 9038 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:23.309908 9697 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:23.310570 9564 consensus_queue.cc:237] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:23.313036 9830 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:23.313396 9434 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:23.314585 9302 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:23.314637 12460 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:23.314793 9038 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:23.315130 9168 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:23.315209 11950 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:23.315431 12460 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:23.315539 12459 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:23.316076 12459 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:23.317747 12459 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:23.319198 8881 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet ded9a2cda7b142d79489c8541e8e15f7 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 14:01:23.319339 9696 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:23.319813 12475 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:23.320852 9168 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:23.320960 9830 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:23.321249 9696 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:23.323442 9301 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:23.323655 8893 catalog_manager.cc:5484] T ded9a2cda7b142d79489c8541e8e15f7 P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:23.324216 9038 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:23.326746 9434 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
W20240502 14:01:23.329176 9501 consensus_peers.cc:485] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca -> Peer 304274b88d234198a3892e38c05014a5 (127.24.66.73:42559): Couldn't send request to peer 304274b88d234198a3892e38c05014a5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ded9a2cda7b142d79489c8541e8e15f7. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:23.431703 12446 ts_tablet_manager.cc:902] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc: Initiating tablet copy from peer d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375)
I20240502 14:01:23.432169 12446 tablet_copy_client.cc:308] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.69:36375
I20240502 14:01:23.436055 9586 tablet_copy_service.cc:140] P d0f45862d7f14ce1976c1a37842f25ca: Received BeginTabletCopySession request for tablet 21d4acfb937f4dd4b992c338fc6b3237 from peer 535b19a469c34efaaf49ca1cc2e1e5fc ({username='slave'} at 127.24.66.74:50525)
I20240502 14:01:23.436195 9586 tablet_copy_service.cc:161] P d0f45862d7f14ce1976c1a37842f25ca: Beginning new tablet copy session on tablet 21d4acfb937f4dd4b992c338fc6b3237 from peer 535b19a469c34efaaf49ca1cc2e1e5fc at {username='slave'} at 127.24.66.74:50525: session id = 535b19a469c34efaaf49ca1cc2e1e5fc-21d4acfb937f4dd4b992c338fc6b3237
I20240502 14:01:23.437438 9586 tablet_copy_source_session.cc:205] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:23.438629 12446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d4acfb937f4dd4b992c338fc6b3237. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:23.440644 12446 tablet_copy_client.cc:783] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:23.440809 12446 tablet_copy_client.cc:647] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:23.442507 12446 tablet_copy_client.cc:523] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:23.443584 12446 tablet_bootstrap.cc:492] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap starting.
I20240502 14:01:23.466325 12446 tablet_bootstrap.cc:492] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:23.466681 12446 tablet_bootstrap.cc:492] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap complete.
I20240502 14:01:23.466874 12446 ts_tablet_manager.cc:1367] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent bootstrapping tablet: real 0.023s user 0.020s sys 0.001s
I20240502 14:01:23.467341 12446 raft_consensus.cc:348] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:23.467494 12446 raft_consensus.cc:729] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 535b19a469c34efaaf49ca1cc2e1e5fc, State: Initialized, Role: LEARNER
I20240502 14:01:23.467615 12446 consensus_queue.cc:260] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:23.467893 12446 ts_tablet_manager.cc:1398] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:23.468529 9586 tablet_copy_service.cc:342] P d0f45862d7f14ce1976c1a37842f25ca: Request end of tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-21d4acfb937f4dd4b992c338fc6b3237 received from {username='slave'} at 127.24.66.74:50525
I20240502 14:01:23.468641 9586 tablet_copy_service.cc:433] P d0f45862d7f14ce1976c1a37842f25ca: ending tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-21d4acfb937f4dd4b992c338fc6b3237 on tablet 21d4acfb937f4dd4b992c338fc6b3237 with peer 535b19a469c34efaaf49ca1cc2e1e5fc
I20240502 14:01:23.501725 11362 raft_consensus.cc:1203] T 6aed24ef77e54c8b8ff09e6709a886e4 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 14:01:23.572912 11362 raft_consensus.cc:1203] T fed16a08a40a474a9159826ca8731acb P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 14:01:23.800138 12597 ts_tablet_manager.cc:902] T ded9a2cda7b142d79489c8541e8e15f7 P 304274b88d234198a3892e38c05014a5: Initiating tablet copy from peer d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375)
I20240502 14:01:23.801050 12597 tablet_copy_client.cc:308] T ded9a2cda7b142d79489c8541e8e15f7 P 304274b88d234198a3892e38c05014a5: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.69:36375
I20240502 14:01:23.804061 11100 raft_consensus.cc:1203] T 6232f47837664d7486612ba1d291f2b8 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 14:01:23.804993 9586 tablet_copy_service.cc:140] P d0f45862d7f14ce1976c1a37842f25ca: Received BeginTabletCopySession request for tablet ded9a2cda7b142d79489c8541e8e15f7 from peer 304274b88d234198a3892e38c05014a5 ({username='slave'} at 127.24.66.73:46083)
I20240502 14:01:23.805128 9586 tablet_copy_service.cc:161] P d0f45862d7f14ce1976c1a37842f25ca: Beginning new tablet copy session on tablet ded9a2cda7b142d79489c8541e8e15f7 from peer 304274b88d234198a3892e38c05014a5 at {username='slave'} at 127.24.66.73:46083: session id = 304274b88d234198a3892e38c05014a5-ded9a2cda7b142d79489c8541e8e15f7
I20240502 14:01:23.806202 9586 tablet_copy_source_session.cc:205] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:23.807518 12597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ded9a2cda7b142d79489c8541e8e15f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:23.809932 12597 tablet_copy_client.cc:783] T ded9a2cda7b142d79489c8541e8e15f7 P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:23.810101 12597 tablet_copy_client.cc:647] T ded9a2cda7b142d79489c8541e8e15f7 P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:23.811229 12597 tablet_copy_client.cc:523] T ded9a2cda7b142d79489c8541e8e15f7 P 304274b88d234198a3892e38c05014a5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:23.813236 12597 tablet_bootstrap.cc:492] T ded9a2cda7b142d79489c8541e8e15f7 P 304274b88d234198a3892e38c05014a5: Bootstrap starting.
I20240502 14:01:23.838248 12597 tablet_bootstrap.cc:492] T ded9a2cda7b142d79489c8541e8e15f7 P 304274b88d234198a3892e38c05014a5: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:23.838624 12597 tablet_bootstrap.cc:492] T ded9a2cda7b142d79489c8541e8e15f7 P 304274b88d234198a3892e38c05014a5: Bootstrap complete.
I20240502 14:01:23.838819 12597 ts_tablet_manager.cc:1367] T ded9a2cda7b142d79489c8541e8e15f7 P 304274b88d234198a3892e38c05014a5: Time spent bootstrapping tablet: real 0.026s user 0.020s sys 0.004s
I20240502 14:01:23.839264 12597 raft_consensus.cc:348] T ded9a2cda7b142d79489c8541e8e15f7 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:23.839432 12597 raft_consensus.cc:729] T ded9a2cda7b142d79489c8541e8e15f7 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 304274b88d234198a3892e38c05014a5, State: Initialized, Role: LEARNER
I20240502 14:01:23.839548 12597 consensus_queue.cc:260] T ded9a2cda7b142d79489c8541e8e15f7 P 304274b88d234198a3892e38c05014a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:23.839902 12597 ts_tablet_manager.cc:1398] T ded9a2cda7b142d79489c8541e8e15f7 P 304274b88d234198a3892e38c05014a5: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 14:01:23.840746 9586 tablet_copy_service.cc:342] P d0f45862d7f14ce1976c1a37842f25ca: Request end of tablet copy session 304274b88d234198a3892e38c05014a5-ded9a2cda7b142d79489c8541e8e15f7 received from {username='slave'} at 127.24.66.73:46083
I20240502 14:01:23.840843 9586 tablet_copy_service.cc:433] P d0f45862d7f14ce1976c1a37842f25ca: ending tablet copy session 304274b88d234198a3892e38c05014a5-ded9a2cda7b142d79489c8541e8e15f7 on tablet ded9a2cda7b142d79489c8541e8e15f7 with peer 304274b88d234198a3892e38c05014a5
I20240502 14:01:23.896046 11362 raft_consensus.cc:1203] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 14:01:23.957674 12576 raft_consensus.cc:1050] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f: attempting to promote NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc to VOTER
I20240502 14:01:23.958228 12576 consensus_queue.cc:237] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:23.960023 9830 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:23.960103 9564 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:23.960268 11362 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:23.960489 9696 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:23.960721 9434 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:23.960693 12624 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:23.960882 12625 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:23.961321 12048 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:23.961479 12625 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:23.961817 12048 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:23.962198 9038 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:23.962705 9168 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:23.963805 12048 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:23.964445 12048 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:23.966562 12625 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:23.967222 9038 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:23.967211 9696 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:23.969023 9434 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:23.969120 9564 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:23.969501 11362 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:23.970865 9830 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:23.972434 8892 catalog_manager.cc:5484] T fed16a08a40a474a9159826ca8731acb P 535b19a469c34efaaf49ca1cc2e1e5fc reported cstate change: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:23.974062 9168 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:23.981246 9302 consensus_queue.cc:237] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:23.983520 9036 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:23.983593 9564 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:23.983880 11362 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:23.983914 9696 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:23.984120 12576 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:23.984277 9830 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:23.984287 12623 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:23.984635 12575 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:23.984863 12048 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:23.985050 12624 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:23.987732 12624 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:23.988018 9168 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:23.988466 9036 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:23.988855 12575 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:23.989632 8880 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet fed16a08a40a474a9159826ca8731acb with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:23.989979 11362 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:23.991719 8889 catalog_manager.cc:5484] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d reported cstate change: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:23.992071 9168 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:23.997138 9696 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:23.999228 9564 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:23.999857 9830 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.003688 9412 tablet_service.cc:1514] Processing DeleteTablet for tablet fed16a08a40a474a9159826ca8731acb with delete_type TABLET_DATA_TOMBSTONED (TS 1a490f170c614ee4900ef16c3287c7de not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:33738
I20240502 14:01:24.004673 12638 tablet_replica.cc:330] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de: stopping tablet replica
I20240502 14:01:24.004997 12638 raft_consensus.cc:2229] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:24.005250 12638 raft_consensus.cc:2258] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:24.005926 12638 ts_tablet_manager.cc:1875] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:24.007956 12638 ts_tablet_manager.cc:1888] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 14:01:24.008029 12638 log.cc:1196] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/fed16a08a40a474a9159826ca8731acb
I20240502 14:01:24.008569 8882 catalog_manager.cc:4830] TS 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907): tablet fed16a08a40a474a9159826ca8731acb (table rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]) successfully deleted
I20240502 14:01:24.105543 12248 raft_consensus.cc:1050] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de: attempting to promote NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc to VOTER
I20240502 14:01:24.106060 12248 consensus_queue.cc:237] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:24.108075 9696 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:24.108075 9302 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:24.108186 9830 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:24.108297 9170 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:24.108506 12248 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:24.108734 11362 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:24.108795 9038 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:24.109064 9564 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:24.109324 12668 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:24.109483 12249 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:24.109639 12667 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:24.109813 12665 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:24.110154 12248 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:24.110339 12669 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:24.112684 12665 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.113204 9302 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.114779 9038 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.115365 9696 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.115365 11362 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.116905 9170 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.118597 8896 catalog_manager.cc:5484] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.119674 9830 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.121578 9564 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.126195 9434 consensus_queue.cc:237] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:24.127753 9170 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:24.127763 11362 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:24.128136 9302 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:24.128217 12249 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:24.128436 12665 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:24.128666 9696 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:24.128724 9038 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:24.129248 12666 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:24.129573 12666 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:24.130352 12249 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:24.131474 12668 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.131714 9830 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:24.132113 12669 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:24.131948 11362 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.132197 9696 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.133212 9830 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.133967 8882 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6aed24ef77e54c8b8ff09e6709a886e4 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:24.134742 8890 catalog_manager.cc:5484] T 6aed24ef77e54c8b8ff09e6709a886e4 P 535b19a469c34efaaf49ca1cc2e1e5fc reported cstate change: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.135463 9038 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.137493 9302 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.137800 9170 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.140165 9544 tablet_service.cc:1514] Processing DeleteTablet for tablet 6aed24ef77e54c8b8ff09e6709a886e4 with delete_type TABLET_DATA_TOMBSTONED (TS d0f45862d7f14ce1976c1a37842f25ca not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:46242
I20240502 14:01:24.140694 12678 tablet_replica.cc:330] T 6aed24ef77e54c8b8ff09e6709a886e4 P d0f45862d7f14ce1976c1a37842f25ca: stopping tablet replica
I20240502 14:01:24.140863 12678 raft_consensus.cc:2229] T 6aed24ef77e54c8b8ff09e6709a886e4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:24.141368 12678 raft_consensus.cc:2258] T 6aed24ef77e54c8b8ff09e6709a886e4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:24.142325 12678 ts_tablet_manager.cc:1875] T 6aed24ef77e54c8b8ff09e6709a886e4 P d0f45862d7f14ce1976c1a37842f25ca: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:24.145649 12678 ts_tablet_manager.cc:1888] T 6aed24ef77e54c8b8ff09e6709a886e4 P d0f45862d7f14ce1976c1a37842f25ca: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 14:01:24.145777 12678 log.cc:1196] T 6aed24ef77e54c8b8ff09e6709a886e4 P d0f45862d7f14ce1976c1a37842f25ca: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal/wals/6aed24ef77e54c8b8ff09e6709a886e4
I20240502 14:01:24.146453 8881 catalog_manager.cc:4830] TS d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375): tablet 6aed24ef77e54c8b8ff09e6709a886e4 (table rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]) successfully deleted
I20240502 14:01:24.233141 12304 raft_consensus.cc:1050] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b: attempting to promote NON_VOTER 304274b88d234198a3892e38c05014a5 to VOTER
I20240502 14:01:24.233690 12304 consensus_queue.cc:237] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:24.235641 9564 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:24.235711 9302 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:24.235780 9696 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:24.235989 9036 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:24.236279 12681 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:24.236414 9170 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:24.236500 12304 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:24.236702 12682 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:24.237573 9434 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:24.238116 12681 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:24.238301 12304 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:24.238345 11100 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:24.238690 12129 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:24.239634 12682 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:24.241909 12129 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:24.242666 11100 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:24.244350 9564 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:24.245070 9696 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:24.245579 8895 catalog_manager.cc:5484] T 6232f47837664d7486612ba1d291f2b8 P 304274b88d234198a3892e38c05014a5 reported cstate change: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:24.245682 9036 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:24.248564 9302 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:24.255481 9434 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:24.255863 9830 consensus_queue.cc:237] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:24.258145 9170 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:24.258518 9036 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:24.258867 11100 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:24.259793 9168 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:24.261406 9434 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:24.261696 9696 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:24.262436 12609 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:24.262621 12304 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:24.262765 12129 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:24.263167 12680 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:24.264379 12682 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:24.269428 12682 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:24.271605 9434 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:24.272519 9696 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:24.272845 8880 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6232f47837664d7486612ba1d291f2b8 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:24.273468 9168 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:24.273663 8889 catalog_manager.cc:5484] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:24.274655 9036 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:24.275214 11100 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:24.276995 9302 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:24.277573 12304 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:24.279137 9301 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:24.279523 9830 consensus_queue.cc:237] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:24.282094 9168 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:24.282137 9564 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:24.282538 9302 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:24.282881 12680 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:24.282943 9696 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:24.283048 9544 tablet_service.cc:1514] Processing DeleteTablet for tablet 6232f47837664d7486612ba1d291f2b8 with delete_type TABLET_DATA_TOMBSTONED (TS d0f45862d7f14ce1976c1a37842f25ca not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:46242
I20240502 14:01:24.283232 12680 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:24.283290 12678 tablet_replica.cc:330] T 6232f47837664d7486612ba1d291f2b8 P d0f45862d7f14ce1976c1a37842f25ca: stopping tablet replica
I20240502 14:01:24.283368 12609 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:24.283424 12678 raft_consensus.cc:2229] T 6232f47837664d7486612ba1d291f2b8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:24.283445 9036 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:24.283622 12678 raft_consensus.cc:2258] T 6232f47837664d7486612ba1d291f2b8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:24.284229 9434 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:24.284667 12678 ts_tablet_manager.cc:1875] T 6232f47837664d7486612ba1d291f2b8 P d0f45862d7f14ce1976c1a37842f25ca: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:24.285539 12129 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:24.285861 12129 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:24.286244 12609 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:24.287376 12678 ts_tablet_manager.cc:1888] T 6232f47837664d7486612ba1d291f2b8 P d0f45862d7f14ce1976c1a37842f25ca: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 14:01:24.287447 12678 log.cc:1196] T 6232f47837664d7486612ba1d291f2b8 P d0f45862d7f14ce1976c1a37842f25ca: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal/wals/6232f47837664d7486612ba1d291f2b8
I20240502 14:01:24.288269 8881 catalog_manager.cc:4830] TS d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375): tablet 6232f47837664d7486612ba1d291f2b8 (table rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]) successfully deleted
I20240502 14:01:24.291209 12680 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:24.291667 9036 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:24.293237 9564 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:24.294342 8892 catalog_manager.cc:5484] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:24.294555 9696 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:24.294711 9302 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:24.297287 9434 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:24.302080 9168 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:24.306660 9830 consensus_queue.cc:237] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:24.308830 9564 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:24.309202 9302 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:24.309280 12609 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:24.309633 9036 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240502 14:01:24.309715 9765 consensus_peers.cc:485] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b -> Peer 304274b88d234198a3892e38c05014a5 (127.24.66.73:42559): Couldn't send request to peer 304274b88d234198a3892e38c05014a5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39bef244549d4bb7a980bf7bc52a01b4. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:24.310040 9434 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:24.310856 12609 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:24.311100 12681 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:24.311267 12609 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:24.311514 9168 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:24.311615 9696 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:24.312013 12681 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:24.312343 12681 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:24.314888 12681 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:24.315476 9301 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:24.316112 9434 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:24.317720 9696 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:24.318024 9036 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:24.318641 8896 catalog_manager.cc:5484] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:24.320695 8880 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 39bef244549d4bb7a980bf7bc52a01b4 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 14:01:24.317716 9564 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:24.325032 9168 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:24.331446 12459 raft_consensus.cc:1050] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca: attempting to promote NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc to VOTER
I20240502 14:01:24.331977 12459 consensus_queue.cc:237] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:24.333853 9830 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:24.333896 11362 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:24.334247 12463 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:24.334619 12460 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:24.336762 9696 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:24.336794 9434 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:24.337153 9168 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:24.337322 12460 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:24.337555 9036 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:24.338035 12463 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:24.338157 9302 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:24.338371 12463 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:24.338932 12460 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:24.341706 12717 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.342206 9301 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.344086 9830 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.344388 12460 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:24.345731 8892 catalog_manager.cc:5484] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.346002 9697 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.348026 9168 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.351142 11362 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.344483 9434 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.356742 9036 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.418754 11100 raft_consensus.cc:1203] T ded9a2cda7b142d79489c8541e8e15f7 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 14:01:24.771456 12756 ts_tablet_manager.cc:902] T 39bef244549d4bb7a980bf7bc52a01b4 P 304274b88d234198a3892e38c05014a5: Initiating tablet copy from peer 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:24.771955 12756 tablet_copy_client.cc:308] T 39bef244549d4bb7a980bf7bc52a01b4 P 304274b88d234198a3892e38c05014a5: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.71:33679
I20240502 14:01:24.772508 9850 tablet_copy_service.cc:140] P 0488b0e4ed994db19d40aed8c15f945b: Received BeginTabletCopySession request for tablet 39bef244549d4bb7a980bf7bc52a01b4 from peer 304274b88d234198a3892e38c05014a5 ({username='slave'} at 127.24.66.73:58115)
I20240502 14:01:24.772639 9850 tablet_copy_service.cc:161] P 0488b0e4ed994db19d40aed8c15f945b: Beginning new tablet copy session on tablet 39bef244549d4bb7a980bf7bc52a01b4 from peer 304274b88d234198a3892e38c05014a5 at {username='slave'} at 127.24.66.73:58115: session id = 304274b88d234198a3892e38c05014a5-39bef244549d4bb7a980bf7bc52a01b4
I20240502 14:01:24.773530 9850 tablet_copy_source_session.cc:205] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:24.774392 12756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39bef244549d4bb7a980bf7bc52a01b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:24.776298 12756 tablet_copy_client.cc:783] T 39bef244549d4bb7a980bf7bc52a01b4 P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:24.776499 12756 tablet_copy_client.cc:647] T 39bef244549d4bb7a980bf7bc52a01b4 P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:24.777719 12756 tablet_copy_client.cc:523] T 39bef244549d4bb7a980bf7bc52a01b4 P 304274b88d234198a3892e38c05014a5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:24.779024 12756 tablet_bootstrap.cc:492] T 39bef244549d4bb7a980bf7bc52a01b4 P 304274b88d234198a3892e38c05014a5: Bootstrap starting.
I20240502 14:01:24.803571 12756 tablet_bootstrap.cc:492] T 39bef244549d4bb7a980bf7bc52a01b4 P 304274b88d234198a3892e38c05014a5: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:24.803898 12756 tablet_bootstrap.cc:492] T 39bef244549d4bb7a980bf7bc52a01b4 P 304274b88d234198a3892e38c05014a5: Bootstrap complete.
I20240502 14:01:24.804068 12756 ts_tablet_manager.cc:1367] T 39bef244549d4bb7a980bf7bc52a01b4 P 304274b88d234198a3892e38c05014a5: Time spent bootstrapping tablet: real 0.025s user 0.022s sys 0.000s
I20240502 14:01:24.804548 12756 raft_consensus.cc:348] T 39bef244549d4bb7a980bf7bc52a01b4 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:24.804672 12756 raft_consensus.cc:729] T 39bef244549d4bb7a980bf7bc52a01b4 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 304274b88d234198a3892e38c05014a5, State: Initialized, Role: LEARNER
I20240502 14:01:24.804787 12756 consensus_queue.cc:260] T 39bef244549d4bb7a980bf7bc52a01b4 P 304274b88d234198a3892e38c05014a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:24.805073 12756 ts_tablet_manager.cc:1398] T 39bef244549d4bb7a980bf7bc52a01b4 P 304274b88d234198a3892e38c05014a5: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 14:01:24.805764 9850 tablet_copy_service.cc:342] P 0488b0e4ed994db19d40aed8c15f945b: Request end of tablet copy session 304274b88d234198a3892e38c05014a5-39bef244549d4bb7a980bf7bc52a01b4 received from {username='slave'} at 127.24.66.73:58115
I20240502 14:01:24.805850 9850 tablet_copy_service.cc:433] P 0488b0e4ed994db19d40aed8c15f945b: ending tablet copy session 304274b88d234198a3892e38c05014a5-39bef244549d4bb7a980bf7bc52a01b4 on tablet 39bef244549d4bb7a980bf7bc52a01b4 with peer 304274b88d234198a3892e38c05014a5
I20240502 14:01:24.943771 9564 consensus_queue.cc:237] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:24.945626 9830 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:24.945924 9696 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:24.945931 11362 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:24.946080 12777 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:24.946280 9301 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:24.946604 9036 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:24.946645 12714 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:24.946754 9168 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:24.946920 12778 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:24.947307 12714 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:24.947507 12778 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:24.947863 12714 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:24.949980 12779 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.950456 9696 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.951373 9036 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.953303 11362 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.953670 8881 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 21d4acfb937f4dd4b992c338fc6b3237 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:24.954463 9301 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.954658 8896 catalog_manager.cc:5484] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca reported cstate change: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:24.954957 9830 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.957541 9168 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:24.958427 9412 tablet_service.cc:1514] Processing DeleteTablet for tablet 21d4acfb937f4dd4b992c338fc6b3237 with delete_type TABLET_DATA_TOMBSTONED (TS 1a490f170c614ee4900ef16c3287c7de not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:33738
I20240502 14:01:24.961743 12786 tablet_replica.cc:330] T 21d4acfb937f4dd4b992c338fc6b3237 P 1a490f170c614ee4900ef16c3287c7de: stopping tablet replica
I20240502 14:01:24.961921 12786 raft_consensus.cc:2229] T 21d4acfb937f4dd4b992c338fc6b3237 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:24.962177 12786 raft_consensus.cc:2258] T 21d4acfb937f4dd4b992c338fc6b3237 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:24.963071 12786 ts_tablet_manager.cc:1875] T 21d4acfb937f4dd4b992c338fc6b3237 P 1a490f170c614ee4900ef16c3287c7de: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:24.965377 12786 ts_tablet_manager.cc:1888] T 21d4acfb937f4dd4b992c338fc6b3237 P 1a490f170c614ee4900ef16c3287c7de: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 14:01:24.965464 12786 log.cc:1196] T 21d4acfb937f4dd4b992c338fc6b3237 P 1a490f170c614ee4900ef16c3287c7de: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/21d4acfb937f4dd4b992c338fc6b3237
I20240502 14:01:24.966032 8882 catalog_manager.cc:4830] TS 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907): tablet 21d4acfb937f4dd4b992c338fc6b3237 (table rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]) successfully deleted
I20240502 14:01:25.022123 12714 raft_consensus.cc:1050] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca: attempting to promote NON_VOTER 304274b88d234198a3892e38c05014a5 to VOTER
I20240502 14:01:25.022789 12714 consensus_queue.cc:237] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:25.025022 9168 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:25.025058 11100 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:25.025161 9434 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:25.025537 12779 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:25.025629 9830 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:25.025652 9036 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:25.025950 12781 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:25.026048 9301 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:25.026288 12781 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:25.026875 12779 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:25.027813 12714 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:25.028108 12714 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:25.028609 9696 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:25.028997 12779 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:25.032114 12803 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.032294 9168 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.034561 9696 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.035321 8892 catalog_manager.cc:5484] T ded9a2cda7b142d79489c8541e8e15f7 P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.035621 9434 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.036566 9036 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.041514 9301 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.055501 9830 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.056169 9564 consensus_queue.cc:237] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:25.057716 11100 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.058493 9696 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:25.058931 9168 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:25.059103 12802 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:25.059727 11100 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:25.060007 12802 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:25.060436 9301 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:25.060451 9036 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:25.060431 12803 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:25.061045 12714 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:25.061475 12802 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:25.062381 9830 raft_consensus.cc:1261] T ded9a2cda7b142d79489c8541e8e15f7 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:25.063112 12803 consensus_queue.cc:1035] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:25.065311 12802 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.066478 9301 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.066495 9696 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.069154 8881 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ded9a2cda7b142d79489c8541e8e15f7 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:25.069557 9036 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.069587 9168 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.071832 8892 catalog_manager.cc:5484] T ded9a2cda7b142d79489c8541e8e15f7 P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.072752 11100 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.077531 9412 tablet_service.cc:1514] Processing DeleteTablet for tablet ded9a2cda7b142d79489c8541e8e15f7 with delete_type TABLET_DATA_TOMBSTONED (TS 1a490f170c614ee4900ef16c3287c7de not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:33738
I20240502 14:01:25.077821 12786 tablet_replica.cc:330] T ded9a2cda7b142d79489c8541e8e15f7 P 1a490f170c614ee4900ef16c3287c7de: stopping tablet replica
I20240502 14:01:25.077986 12786 raft_consensus.cc:2229] T ded9a2cda7b142d79489c8541e8e15f7 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:25.078217 12786 raft_consensus.cc:2258] T ded9a2cda7b142d79489c8541e8e15f7 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:25.080060 12786 ts_tablet_manager.cc:1875] T ded9a2cda7b142d79489c8541e8e15f7 P 1a490f170c614ee4900ef16c3287c7de: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:25.079836 9830 raft_consensus.cc:2932] T ded9a2cda7b142d79489c8541e8e15f7 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.082309 12786 ts_tablet_manager.cc:1888] T ded9a2cda7b142d79489c8541e8e15f7 P 1a490f170c614ee4900ef16c3287c7de: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 14:01:25.082397 12786 log.cc:1196] T ded9a2cda7b142d79489c8541e8e15f7 P 1a490f170c614ee4900ef16c3287c7de: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/ded9a2cda7b142d79489c8541e8e15f7
I20240502 14:01:25.082953 8882 catalog_manager.cc:4830] TS 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907): tablet ded9a2cda7b142d79489c8541e8e15f7 (table rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]) successfully deleted
I20240502 14:01:25.175927 11100 raft_consensus.cc:1203] T 39bef244549d4bb7a980bf7bc52a01b4 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 14:01:25.238999 9301 consensus_queue.cc:237] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:25.240798 9830 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:25.240995 9168 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:25.240995 9434 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:25.241230 12821 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:25.241621 12822 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:25.242025 12821 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:25.242318 9696 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:25.242872 9564 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:25.242954 12822 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:25.243129 9036 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:25.243276 12819 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:25.243593 12822 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:25.245249 12819 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:25.245692 9564 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:25.247658 9830 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:25.249423 9434 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:25.249935 8895 catalog_manager.cc:5484] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:25.251456 9168 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:25.255465 9696 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:25.258217 9036 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:25.262775 9301 consensus_queue.cc:237] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:25.265517 9696 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
W20240502 14:01:25.265564 9238 consensus_peers.cc:485] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f -> Peer e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72:35005): Couldn't send request to peer e01cf117ef5d446590c06dd1a6ab7ef7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0801ed14be134f2a97d33a5c7a8f5cff. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:25.266168 12820 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:25.266486 9564 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:25.267170 12835 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:25.267719 9830 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:25.268630 9434 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:25.268784 12820 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:25.270201 12835 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:25.270368 9168 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:25.270745 9036 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:25.271023 12740 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:25.270931 12820 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:25.271195 12822 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:25.272590 9168 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:25.272943 9038 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:25.273311 8880 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 0801ed14be134f2a97d33a5c7a8f5cff with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 14:01:25.273597 9564 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:25.273653 9696 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:25.276865 8896 catalog_manager.cc:5484] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:25.276841 9828 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:25.279469 9434 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:25.507349 9564 consensus_queue.cc:237] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } }
I20240502 14:01:25.509130 9038 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:25.509438 9301 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:25.509582 9168 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:25.509608 12802 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:25.509953 12779 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:25.510001 9696 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:25.510376 9434 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:25.510757 9828 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:25.511142 12802 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:25.511473 12802 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:25.512821 12802 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:25.513167 12779 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:25.513514 12779 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:25.513950 9696 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:25.514705 9830 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:25.516849 8889 catalog_manager.cc:5484] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } }
I20240502 14:01:25.517184 9038 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:25.518113 9301 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:25.519882 9434 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:25.524761 9564 consensus_queue.cc:237] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:25.525664 9168 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } }
I20240502 14:01:25.527056 9301 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:25.527170 9696 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:25.527472 12803 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:25.527529 9830 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:25.527961 12803 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:25.528052 12714 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:25.528874 9434 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:25.529594 9168 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:25.529834 12781 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:25.529958 12803 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:25.530411 9038 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:25.531858 12803 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20240502 14:01:25.532308 9502 consensus_peers.cc:485] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca -> Peer e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72:35005): Couldn't send request to peer e01cf117ef5d446590c06dd1a6ab7ef7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7289a5be3b3f437f9a2770baa42b89a8. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:25.532313 12803 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:25.532672 9168 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:25.534453 8881 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 7289a5be3b3f437f9a2770baa42b89a8 with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 14:01:25.534729 9301 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:25.535499 9696 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:25.535931 8889 catalog_manager.cc:5484] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:25.536787 9830 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:25.541079 9434 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:25.544000 9038 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:25.636726 12681 raft_consensus.cc:1050] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b: attempting to promote NON_VOTER 304274b88d234198a3892e38c05014a5 to VOTER
I20240502 14:01:25.637295 12681 consensus_queue.cc:237] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:25.639160 9696 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:25.639431 11100 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:25.639501 9168 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:25.639537 9301 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:25.639899 12681 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:25.640092 12609 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:25.640726 12681 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:25.640920 12878 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:25.641973 9564 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:25.642027 9434 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:25.642379 12878 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:25.642685 12681 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:25.648618 9038 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:25.649219 12681 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:25.650427 12609 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.651495 9168 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.651907 9564 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.653168 9696 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.653595 9301 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.655145 9434 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.656203 8890 catalog_manager.cc:5484] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:25.656692 11100 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.661226 9038 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.667965 9830 consensus_queue.cc:237] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:25.670128 9301 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:25.670116 9038 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:25.670158 9696 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:25.670158 9564 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:25.670553 12877 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:25.670682 9168 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:25.670948 11100 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:25.670933 12877 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:25.671442 12877 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:25.671643 12696 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:25.671779 12877 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:25.671900 12888 ts_tablet_manager.cc:902] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7: Initiating tablet copy from peer ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:25.671900 12681 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:25.672461 12888 tablet_copy_client.cc:308] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.67:34749
I20240502 14:01:25.673290 9322 tablet_copy_service.cc:140] P ccfb5bbc1bb54793812d11170ac2d25f: Received BeginTabletCopySession request for tablet 0801ed14be134f2a97d33a5c7a8f5cff from peer e01cf117ef5d446590c06dd1a6ab7ef7 ({username='slave'} at 127.24.66.72:33161)
I20240502 14:01:25.673398 9322 tablet_copy_service.cc:161] P ccfb5bbc1bb54793812d11170ac2d25f: Beginning new tablet copy session on tablet 0801ed14be134f2a97d33a5c7a8f5cff from peer e01cf117ef5d446590c06dd1a6ab7ef7 at {username='slave'} at 127.24.66.72:33161: session id = e01cf117ef5d446590c06dd1a6ab7ef7-0801ed14be134f2a97d33a5c7a8f5cff
I20240502 14:01:25.674382 9322 tablet_copy_source_session.cc:205] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:25.674530 12609 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.675755 9696 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.676040 9168 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.676834 9564 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.675446 12888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0801ed14be134f2a97d33a5c7a8f5cff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:25.678555 11100 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.679255 8890 catalog_manager.cc:5484] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288 reported cstate change: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.680482 9038 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.681442 12888 tablet_copy_client.cc:783] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:25.681742 12888 tablet_copy_client.cc:647] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:25.681919 8880 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 39bef244549d4bb7a980bf7bc52a01b4 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:25.682679 9301 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:25.683743 9412 tablet_service.cc:1514] Processing DeleteTablet for tablet 39bef244549d4bb7a980bf7bc52a01b4 with delete_type TABLET_DATA_TOMBSTONED (TS 1a490f170c614ee4900ef16c3287c7de not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:33738
I20240502 14:01:25.684042 12888 tablet_copy_client.cc:523] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:25.686367 12888 tablet_bootstrap.cc:492] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap starting.
I20240502 14:01:25.686865 12893 tablet_replica.cc:330] T 39bef244549d4bb7a980bf7bc52a01b4 P 1a490f170c614ee4900ef16c3287c7de: stopping tablet replica
I20240502 14:01:25.687047 12893 raft_consensus.cc:2229] T 39bef244549d4bb7a980bf7bc52a01b4 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:25.687251 12893 raft_consensus.cc:2258] T 39bef244549d4bb7a980bf7bc52a01b4 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:25.687850 12893 ts_tablet_manager.cc:1875] T 39bef244549d4bb7a980bf7bc52a01b4 P 1a490f170c614ee4900ef16c3287c7de: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:25.689679 12893 ts_tablet_manager.cc:1888] T 39bef244549d4bb7a980bf7bc52a01b4 P 1a490f170c614ee4900ef16c3287c7de: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 14:01:25.689751 12893 log.cc:1196] T 39bef244549d4bb7a980bf7bc52a01b4 P 1a490f170c614ee4900ef16c3287c7de: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/39bef244549d4bb7a980bf7bc52a01b4
I20240502 14:01:25.690322 8882 catalog_manager.cc:4830] TS 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907): tablet 39bef244549d4bb7a980bf7bc52a01b4 (table rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]) successfully deleted
I20240502 14:01:25.705842 12888 tablet_bootstrap.cc:492] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:25.706111 12888 tablet_bootstrap.cc:492] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap complete.
I20240502 14:01:25.706259 12888 ts_tablet_manager.cc:1367] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent bootstrapping tablet: real 0.020s user 0.016s sys 0.002s
I20240502 14:01:25.706629 12888 raft_consensus.cc:348] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:25.706745 12888 raft_consensus.cc:729] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e01cf117ef5d446590c06dd1a6ab7ef7, State: Initialized, Role: LEARNER
I20240502 14:01:25.706825 12888 consensus_queue.cc:260] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:25.707154 12888 ts_tablet_manager.cc:1398] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent starting tablet: real 0.001s user 0.002s sys 0.001s
I20240502 14:01:25.707695 9322 tablet_copy_service.cc:342] P ccfb5bbc1bb54793812d11170ac2d25f: Request end of tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-0801ed14be134f2a97d33a5c7a8f5cff received from {username='slave'} at 127.24.66.72:33161
I20240502 14:01:25.707772 9322 tablet_copy_service.cc:433] P ccfb5bbc1bb54793812d11170ac2d25f: ending tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-0801ed14be134f2a97d33a5c7a8f5cff on tablet 0801ed14be134f2a97d33a5c7a8f5cff with peer e01cf117ef5d446590c06dd1a6ab7ef7
I20240502 14:01:25.988129 9830 consensus_queue.cc:237] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } }
I20240502 14:01:25.990129 9434 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:25.990164 9301 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:25.990193 9168 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:25.990433 9564 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:25.990713 9696 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:25.990990 12609 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:25.991179 12681 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:25.991375 12696 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:25.992112 9038 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 14:01:25.994072 12696 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:25.994499 9564 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:25.996661 9696 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:25.996750 9301 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:25.997191 8890 catalog_manager.cc:5484] T 189b91e40d604414ab0497058929777c P d0f45862d7f14ce1976c1a37842f25ca reported cstate change: config changed from index -1 to 4. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:25.997395 12681 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:25.997721 12681 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:25.998011 12681 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 14:01:25.999579 9434 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:26.000069 9038 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:26.000821 9168 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } }
I20240502 14:01:26.008922 9830 consensus_queue.cc:237] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:26.010722 9168 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:26.010787 9038 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:26.011449 9301 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:26.011464 9564 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:26.011837 9696 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:26.011888 12681 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:26.012091 9434 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 14:01:26.012185 12681 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:26.012653 12879 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:26.013180 12879 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:26.013463 12879 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20240502 14:01:26.014714 9765 consensus_peers.cc:485] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b -> Peer 304274b88d234198a3892e38c05014a5 (127.24.66.73:42559): Couldn't send request to peer 304274b88d234198a3892e38c05014a5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 189b91e40d604414ab0497058929777c. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:26.014837 12878 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 14:01:26.015288 12681 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:26.015679 9564 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:26.017688 9696 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:26.018103 9301 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:26.018838 8893 catalog_manager.cc:5484] T 189b91e40d604414ab0497058929777c P d0f45862d7f14ce1976c1a37842f25ca reported cstate change: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:26.020370 12888 ts_tablet_manager.cc:902] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7: Initiating tablet copy from peer d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375)
I20240502 14:01:26.020843 12888 tablet_copy_client.cc:308] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.69:36375
I20240502 14:01:26.022531 8880 catalog_manager.cc:4997] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 189b91e40d604414ab0497058929777c with cas_config_opid_index 4: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20240502 14:01:26.023097 9168 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:26.023944 9434 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:26.024895 9038 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:26.038400 9586 tablet_copy_service.cc:140] P d0f45862d7f14ce1976c1a37842f25ca: Received BeginTabletCopySession request for tablet 7289a5be3b3f437f9a2770baa42b89a8 from peer e01cf117ef5d446590c06dd1a6ab7ef7 ({username='slave'} at 127.24.66.72:45915)
I20240502 14:01:26.038502 9586 tablet_copy_service.cc:161] P d0f45862d7f14ce1976c1a37842f25ca: Beginning new tablet copy session on tablet 7289a5be3b3f437f9a2770baa42b89a8 from peer e01cf117ef5d446590c06dd1a6ab7ef7 at {username='slave'} at 127.24.66.72:45915: session id = e01cf117ef5d446590c06dd1a6ab7ef7-7289a5be3b3f437f9a2770baa42b89a8
I20240502 14:01:26.039489 9586 tablet_copy_source_session.cc:205] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:26.040994 12888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7289a5be3b3f437f9a2770baa42b89a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:26.043112 12888 tablet_copy_client.cc:783] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:26.043249 12888 tablet_copy_client.cc:647] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:26.046727 12888 tablet_copy_client.cc:523] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:26.047747 12888 tablet_bootstrap.cc:492] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap starting.
I20240502 14:01:26.069175 12888 tablet_bootstrap.cc:492] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:26.069451 12888 tablet_bootstrap.cc:492] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap complete.
I20240502 14:01:26.069620 12888 ts_tablet_manager.cc:1367] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent bootstrapping tablet: real 0.022s user 0.021s sys 0.000s
I20240502 14:01:26.070021 12888 raft_consensus.cc:348] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:26.070183 12888 raft_consensus.cc:729] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e01cf117ef5d446590c06dd1a6ab7ef7, State: Initialized, Role: LEARNER
I20240502 14:01:26.070278 12888 consensus_queue.cc:260] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:26.070535 12888 ts_tablet_manager.cc:1398] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:26.071264 9586 tablet_copy_service.cc:342] P d0f45862d7f14ce1976c1a37842f25ca: Request end of tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-7289a5be3b3f437f9a2770baa42b89a8 received from {username='slave'} at 127.24.66.72:45915
I20240502 14:01:26.071352 9586 tablet_copy_service.cc:433] P d0f45862d7f14ce1976c1a37842f25ca: ending tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-7289a5be3b3f437f9a2770baa42b89a8 on tablet 7289a5be3b3f437f9a2770baa42b89a8 with peer e01cf117ef5d446590c06dd1a6ab7ef7
I20240502 14:01:26.221254 11231 raft_consensus.cc:1203] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 14:01:26.573228 11231 raft_consensus.cc:1203] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 14:01:26.583328 12977 ts_tablet_manager.cc:902] T 189b91e40d604414ab0497058929777c P 304274b88d234198a3892e38c05014a5: Initiating tablet copy from peer 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:26.583884 12977 tablet_copy_client.cc:308] T 189b91e40d604414ab0497058929777c P 304274b88d234198a3892e38c05014a5: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.71:33679
I20240502 14:01:26.584442 9850 tablet_copy_service.cc:140] P 0488b0e4ed994db19d40aed8c15f945b: Received BeginTabletCopySession request for tablet 189b91e40d604414ab0497058929777c from peer 304274b88d234198a3892e38c05014a5 ({username='slave'} at 127.24.66.73:58115)
I20240502 14:01:26.584548 9850 tablet_copy_service.cc:161] P 0488b0e4ed994db19d40aed8c15f945b: Beginning new tablet copy session on tablet 189b91e40d604414ab0497058929777c from peer 304274b88d234198a3892e38c05014a5 at {username='slave'} at 127.24.66.73:58115: session id = 304274b88d234198a3892e38c05014a5-189b91e40d604414ab0497058929777c
I20240502 14:01:26.585373 9850 tablet_copy_source_session.cc:205] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:26.586306 12977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 189b91e40d604414ab0497058929777c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:26.588387 12977 tablet_copy_client.cc:783] T 189b91e40d604414ab0497058929777c P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:26.588577 12977 tablet_copy_client.cc:647] T 189b91e40d604414ab0497058929777c P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:26.589761 12977 tablet_copy_client.cc:523] T 189b91e40d604414ab0497058929777c P 304274b88d234198a3892e38c05014a5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:26.590894 12977 tablet_bootstrap.cc:492] T 189b91e40d604414ab0497058929777c P 304274b88d234198a3892e38c05014a5: Bootstrap starting.
I20240502 14:01:26.610528 12977 tablet_bootstrap.cc:492] T 189b91e40d604414ab0497058929777c P 304274b88d234198a3892e38c05014a5: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:26.610872 12977 tablet_bootstrap.cc:492] T 189b91e40d604414ab0497058929777c P 304274b88d234198a3892e38c05014a5: Bootstrap complete.
I20240502 14:01:26.611084 12977 ts_tablet_manager.cc:1367] T 189b91e40d604414ab0497058929777c P 304274b88d234198a3892e38c05014a5: Time spent bootstrapping tablet: real 0.020s user 0.019s sys 0.000s
I20240502 14:01:26.611598 12977 raft_consensus.cc:348] T 189b91e40d604414ab0497058929777c P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:26.611714 12977 raft_consensus.cc:729] T 189b91e40d604414ab0497058929777c P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 304274b88d234198a3892e38c05014a5, State: Initialized, Role: LEARNER
I20240502 14:01:26.611819 12977 consensus_queue.cc:260] T 189b91e40d604414ab0497058929777c P 304274b88d234198a3892e38c05014a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:26.612214 12977 ts_tablet_manager.cc:1398] T 189b91e40d604414ab0497058929777c P 304274b88d234198a3892e38c05014a5: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 14:01:26.612774 9850 tablet_copy_service.cc:342] P 0488b0e4ed994db19d40aed8c15f945b: Request end of tablet copy session 304274b88d234198a3892e38c05014a5-189b91e40d604414ab0497058929777c received from {username='slave'} at 127.24.66.73:58115
I20240502 14:01:26.612869 9850 tablet_copy_service.cc:433] P 0488b0e4ed994db19d40aed8c15f945b: ending tablet copy session 304274b88d234198a3892e38c05014a5-189b91e40d604414ab0497058929777c on tablet 189b91e40d604414ab0497058929777c with peer 304274b88d234198a3892e38c05014a5
I20240502 14:01:26.671798 12740 raft_consensus.cc:1050] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f: attempting to promote NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 to VOTER
I20240502 14:01:26.672358 12740 consensus_queue.cc:237] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } }
I20240502 14:01:26.674293 9830 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:26.674520 9696 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:26.675000 12985 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:26.675083 9038 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:26.675298 9168 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:26.675459 12985 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:26.675582 11231 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:26.675601 12823 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:26.676367 9434 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:26.676518 9564 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:26.676774 12987 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:26.676995 12985 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:26.677179 12986 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:26.677542 12986 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:26.679788 12987 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:26.680373 9168 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:26.680922 9564 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:26.681811 11231 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:26.682411 9830 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:26.683437 9038 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:26.684669 8893 catalog_manager.cc:5484] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca reported cstate change: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:26.686671 9696 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:26.687048 9434 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:26.691489 9301 consensus_queue.cc:237] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } }
I20240502 14:01:26.693219 9564 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:26.693465 9168 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:26.693495 9696 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:26.693504 9038 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:26.693840 11231 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:26.693893 12985 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:26.694219 12823 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:26.694563 12823 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:26.694808 12740 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:26.694974 12823 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:26.695892 9830 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:26.696265 12986 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:26.698074 12823 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:26.699244 9696 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:26.700557 8880 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 0801ed14be134f2a97d33a5c7a8f5cff with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:26.701041 9830 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:26.701910 9564 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:26.703660 11231 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:26.703915 8889 catalog_manager.cc:5484] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:26.706001 9168 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:26.706643 9038 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:26.707744 9412 tablet_service.cc:1514] Processing DeleteTablet for tablet 0801ed14be134f2a97d33a5c7a8f5cff with delete_type TABLET_DATA_TOMBSTONED (TS 1a490f170c614ee4900ef16c3287c7de not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:33738
I20240502 14:01:26.711553 13005 tablet_replica.cc:330] T 0801ed14be134f2a97d33a5c7a8f5cff P 1a490f170c614ee4900ef16c3287c7de: stopping tablet replica
I20240502 14:01:26.712548 13005 raft_consensus.cc:2229] T 0801ed14be134f2a97d33a5c7a8f5cff P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:26.713589 13005 raft_consensus.cc:2258] T 0801ed14be134f2a97d33a5c7a8f5cff P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:26.714274 13005 ts_tablet_manager.cc:1875] T 0801ed14be134f2a97d33a5c7a8f5cff P 1a490f170c614ee4900ef16c3287c7de: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:26.716472 13005 ts_tablet_manager.cc:1888] T 0801ed14be134f2a97d33a5c7a8f5cff P 1a490f170c614ee4900ef16c3287c7de: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 14:01:26.716548 13005 log.cc:1196] T 0801ed14be134f2a97d33a5c7a8f5cff P 1a490f170c614ee4900ef16c3287c7de: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/0801ed14be134f2a97d33a5c7a8f5cff
I20240502 14:01:26.717095 8882 catalog_manager.cc:4830] TS 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907): tablet 0801ed14be134f2a97d33a5c7a8f5cff (table rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]) successfully deleted
I20240502 14:01:27.089787 13027 raft_consensus.cc:1050] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca: attempting to promote NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 to VOTER
I20240502 14:01:27.090323 13027 consensus_queue.cc:237] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } }
I20240502 14:01:27.091876 11231 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:27.092109 9301 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:27.092183 9036 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:27.092532 13030 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:27.092643 9168 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:27.092670 9696 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:27.092671 9830 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:27.092998 13033 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:27.093372 13027 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:27.093533 12976 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:27.093688 13031 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:27.094326 9434 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:27.095089 13033 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:27.095499 13033 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:27.096925 13027 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:27.097544 9301 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:27.098203 9830 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:27.098285 9168 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:27.100432 9696 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:27.100841 8894 catalog_manager.cc:5484] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca reported cstate change: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:27.100931 11230 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:27.102950 9036 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:27.105693 9564 consensus_queue.cc:237] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } }
I20240502 14:01:27.107635 9696 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:27.107635 9168 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:27.107789 9830 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:27.108012 13030 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:27.108359 9036 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:27.108400 13030 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:27.109252 13032 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:27.109802 13031 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:27.110342 11230 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:27.111085 9301 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:27.111483 13032 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:27.111630 13031 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:27.111465 13029 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:27.112349 9830 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:27.112632 11230 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:27.112664 9301 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:27.113979 9168 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:27.114236 9036 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:27.114874 8881 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 7289a5be3b3f437f9a2770baa42b89a8 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:27.116489 8892 catalog_manager.cc:5484] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7 reported cstate change: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:27.116976 9696 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:27.119362 9412 tablet_service.cc:1514] Processing DeleteTablet for tablet 7289a5be3b3f437f9a2770baa42b89a8 with delete_type TABLET_DATA_TOMBSTONED (TS 1a490f170c614ee4900ef16c3287c7de not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:33738
I20240502 14:01:27.119601 13005 tablet_replica.cc:330] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de: stopping tablet replica
I20240502 14:01:27.119729 13005 raft_consensus.cc:2229] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:27.119904 13005 pending_rounds.cc:62] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de: Trying to abort 1 pending ops.
I20240502 14:01:27.119973 13005 pending_rounds.cc:68] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de: Aborting op as it isn't in flight: id { term: 1 index: 6 } timestamp: 7023241163120652288 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "7289a5be3b3f437f9a2770baa42b89a8" old_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } } new_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } } }
I20240502 14:01:27.120162 13005 raft_consensus.cc:2894] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Aborting config change with OpId 1.6: Aborted: Op aborted
I20240502 14:01:27.120220 13005 raft_consensus.cc:2866] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20240502 14:01:27.120254 13005 raft_consensus.cc:2258] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:27.120949 13005 ts_tablet_manager.cc:1875] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:27.123320 13005 ts_tablet_manager.cc:1888] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 14:01:27.123411 13005 log.cc:1196] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/7289a5be3b3f437f9a2770baa42b89a8
I20240502 14:01:27.124111 8882 catalog_manager.cc:4830] TS 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907): tablet 7289a5be3b3f437f9a2770baa42b89a8 (table rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]) successfully deleted
I20240502 14:01:27.130012 11100 raft_consensus.cc:1203] T 189b91e40d604414ab0497058929777c P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.4->[1.5-1.5] Dedup: 1.5->[]
I20240502 14:01:27.576987 13042 raft_consensus.cc:1050] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b: attempting to promote NON_VOTER 304274b88d234198a3892e38c05014a5 to VOTER
I20240502 14:01:27.577522 13042 consensus_queue.cc:237] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:27.579355 9302 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:27.579452 9564 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:27.579605 9434 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:27.579792 11100 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:27.579937 9170 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:27.580039 13067 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:27.580235 13068 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:27.580276 9036 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:27.580399 13042 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:27.580798 13069 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:27.581002 9696 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 14:01:27.582857 13068 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:27.583091 13042 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:27.583628 13041 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 14:01:27.584463 13042 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:27.586089 11100 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:27.587237 9170 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:27.587836 9434 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:27.587883 9566 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:27.588060 9036 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:27.589485 8889 catalog_manager.cc:5484] T 189b91e40d604414ab0497058929777c P 304274b88d234198a3892e38c05014a5 reported cstate change: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:27.592168 9302 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:27.592517 9696 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:27.599880 9830 consensus_queue.cc:237] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:27.601557 9564 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:27.601787 11100 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:27.602008 9696 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:27.602226 13077 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:27.602433 13042 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:27.602489 9302 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:27.603032 13077 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:27.603356 13041 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:27.604763 9038 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:27.604775 9170 raft_consensus.cc:1261] T 189b91e40d604414ab0497058929777c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 14:01:27.605258 13041 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:27.605410 13042 consensus_queue.cc:1035] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 14:01:27.606657 13041 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:27.608479 9696 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:27.608496 9564 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:27.608525 9302 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:27.609548 8880 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 189b91e40d604414ab0497058929777c with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:27.609998 11100 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:27.610185 9038 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:27.610388 8888 catalog_manager.cc:5484] T 189b91e40d604414ab0497058929777c P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:27.613377 9412 tablet_service.cc:1514] Processing DeleteTablet for tablet 189b91e40d604414ab0497058929777c with delete_type TABLET_DATA_TOMBSTONED (TS 1a490f170c614ee4900ef16c3287c7de not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:33738
I20240502 14:01:27.613631 13005 tablet_replica.cc:330] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de: stopping tablet replica
I20240502 14:01:27.613770 13005 raft_consensus.cc:2229] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:27.613965 13005 raft_consensus.cc:2258] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:27.614656 13005 ts_tablet_manager.cc:1875] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:27.615523 9170 raft_consensus.cc:2932] T 189b91e40d604414ab0497058929777c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:27.616571 13005 ts_tablet_manager.cc:1888] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 14:01:27.616657 13005 log.cc:1196] T 189b91e40d604414ab0497058929777c P 1a490f170c614ee4900ef16c3287c7de: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-3/wal/wals/189b91e40d604414ab0497058929777c
I20240502 14:01:27.617744 8882 catalog_manager.cc:4830] TS 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907): tablet 189b91e40d604414ab0497058929777c (table rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]) successfully deleted
I20240502 14:01:28.041162 9282 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:01:28.041241 11080 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:28.042538 9544 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:01:28.048485 11342 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:28.049293 9018 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:01:28.082998 9678 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:28.086390 9150 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:01:28.090582 9809 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:01:28.092310 11211 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:28.092440 9412 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:01:28.212527 9412 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:01:28.222057 9544 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:01:28.224347 9282 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:01:28.228899 9678 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:28.229404 11342 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:28.230513 11080 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:28.233191 9018 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:01:28.242947 11211 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:28.251648 9809 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:01:28.274788 9150 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:01:28.329973 9170 consensus_queue.cc:237] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:28.332046 11230 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.332047 9564 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.332263 9038 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.332500 9696 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.332805 13177 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.332891 9828 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.333241 13178 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.333379 11990 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.333395 9302 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.333639 13016 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.334933 13016 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.336576 13177 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.337095 11230 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.337177 13179 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.337620 9302 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.339414 9828 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.339581 9038 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.340230 8889 catalog_manager.cc:5484] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7 reported cstate change: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.340960 9564 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.341907 9696 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
W20240502 14:01:28.347222 9105 consensus_peers.cc:485] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 -> Peer 304274b88d234198a3892e38c05014a5 (127.24.66.73:42559): Couldn't send request to peer 304274b88d234198a3892e38c05014a5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ada4c2cc1749411d8f41f882b7c64b07. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:28.355048 9302 consensus_queue.cc:237] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:28.357124 9038 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.357272 11100 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240502 14:01:28.357424 9238 consensus_peers.cc:485] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f -> Peer e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72:35005): Couldn't send request to peer e01cf117ef5d446590c06dd1a6ab7ef7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bfa2e6af0fd544fa8bfdaefedabb5e68. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:28.357527 9828 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.357548 13157 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.357837 9170 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.358003 13157 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.358206 9564 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.358645 13199 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.359539 9696 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.359761 13055 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.359974 13157 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.360296 13157 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.360847 13201 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.361793 9170 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.364250 9828 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.364871 8894 catalog_manager.cc:5484] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.366879 11099 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.366878 9696 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.367132 9036 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.368575 9564 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.378755 9434 consensus_queue.cc:237] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:28.380710 9830 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.380822 9036 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.380846 9170 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.380868 9696 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240502 14:01:28.381063 9370 consensus_peers.cc:485] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de -> Peer 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74:34421): Couldn't send request to peer 535b19a469c34efaaf49ca1cc2e1e5fc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9eef89d7e129492bab305a1b5009aace. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:28.381256 11099 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.381273 13054 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.381532 9302 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.381604 12666 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.381937 12666 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.382114 13217 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.382344 13054 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.384024 13216 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.384951 13215 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:28.385859 9302 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:28.386512 9830 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:28.387542 9696 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:28.389427 8896 catalog_manager.cc:5484] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 7 to 8, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:28.390674 11099 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:28.390815 9170 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:28.391561 9036 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:28.402221 9564 consensus_queue.cc:237] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:28.403947 11230 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.404095 9036 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.404479 9170 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.404531 9302 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.404731 9830 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.404750 9696 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.404938 13029 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.405308 13029 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.405580 13230 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.405731 13029 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.406013 13029 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.406293 13029 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240502 14:01:28.407783 9501 consensus_peers.cc:485] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca -> Peer 304274b88d234198a3892e38c05014a5 (127.24.66.73:42559): Couldn't send request to peer 304274b88d234198a3892e38c05014a5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7289a5be3b3f437f9a2770baa42b89a8. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:28.409013 13230 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.409588 9036 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.409901 9830 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.411747 9696 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.412678 9301 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.415503 8890 catalog_manager.cc:5484] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.416760 9170 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.421125 11230 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.428666 9830 consensus_queue.cc:237] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:28.430430 9434 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.431161 11099 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.431234 13069 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.431613 9170 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.431995 9301 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.432372 13068 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.432731 13068 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.433138 13068 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.433205 9036 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.433655 13069 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.434288 9696 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.434741 13068 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.437196 13068 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.437747 9036 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.439440 9434 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.440771 9696 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.441249 8892 catalog_manager.cc:5484] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d reported cstate change: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.446094 9301 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.446358 11099 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.447038 9170 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
W20240502 14:01:28.450448 9766 consensus_peers.cc:485] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b -> Peer e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72:35005): Couldn't send request to peer e01cf117ef5d446590c06dd1a6ab7ef7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 97fa16722a1f4adcbf8a5dc3fdfab77a. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:28.462914 9830 consensus_queue.cc:237] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:28.465001 9170 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.465060 9564 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.465276 9696 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240502 14:01:28.465282 9766 consensus_peers.cc:485] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b -> Peer 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74:34421): Couldn't send request to peer 535b19a469c34efaaf49ca1cc2e1e5fc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39bef244549d4bb7a980bf7bc52a01b4. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:28.465627 9036 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.465616 13250 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.465902 9301 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.466215 13248 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.466354 13250 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.466457 13069 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.466681 11099 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.466786 13247 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.467057 13246 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.469259 13246 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:28.469709 9170 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:28.471673 11099 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:28.471966 9564 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:28.472488 8888 catalog_manager.cc:5484] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: config changed from index 7 to 8, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:28.477731 9696 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:28.478063 9301 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:28.485509 9434 consensus_queue.cc:237] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:28.487455 11230 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.487470 9170 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.487522 9301 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.487974 13216 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.488380 13216 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.488111 9036 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:28.489784 9035 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240502 14:01:28.490160 9369 consensus_peers.cc:485] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de -> Peer 304274b88d234198a3892e38c05014a5 (127.24.66.73:42559): Couldn't send request to peer 304274b88d234198a3892e38c05014a5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8c10fb9748c747c6a89066fa113cda3c. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:28.490342 13216 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.491168 9696 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.491650 13217 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.491811 13216 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.493763 9830 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.494104 13217 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.494508 13216 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.495314 11230 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.497334 9830 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.497336 9301 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.498642 8896 catalog_manager.cc:5484] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de reported cstate change: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:28.500444 9035 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.500845 9170 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.503306 9696 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.510793 9830 consensus_queue.cc:237] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:28.512771 9035 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.512746 9696 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.512990 9170 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.513270 9301 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.513331 13069 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.513391 11099 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240502 14:01:28.513796 9766 consensus_peers.cc:485] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b -> Peer e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72:35005): Couldn't send request to peer e01cf117ef5d446590c06dd1a6ab7ef7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6232f47837664d7486612ba1d291f2b8. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:28.513837 13069 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.514017 13246 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.514125 13068 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.515043 13248 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.515122 9434 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.516346 13246 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.516870 13246 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.518168 9035 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.518625 11099 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.519791 9696 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.521193 8892 catalog_manager.cc:5484] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:28.523478 9301 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.525815 9434 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.526991 9168 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.533529 9434 consensus_queue.cc:237] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:28.535271 9301 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.535277 9830 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.535662 9035 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.535668 11230 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240502 14:01:28.536069 9369 consensus_peers.cc:485] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de -> Peer 304274b88d234198a3892e38c05014a5 (127.24.66.73:42559): Couldn't send request to peer 304274b88d234198a3892e38c05014a5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b1548ca1bcd24d288d30117baea53c12. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:28.536240 9168 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.536214 13217 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.536304 9696 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.536403 12666 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.536557 13215 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.536716 13054 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.536994 13054 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.538221 13217 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.540360 13215 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.540989 9301 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.541433 9168 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.543237 11230 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.543960 9830 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.544441 8892 catalog_manager.cc:5484] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de reported cstate change: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:28.546609 9696 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.551756 9035 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.556699 9434 consensus_queue.cc:237] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:28.558396 9035 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.558399 9168 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.558530 9830 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.558876 13301 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.558941 9696 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.559068 13217 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.559309 13217 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.559451 11362 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.559866 13301 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.560294 9301 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.560595 13301 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.560806 13215 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240502 14:01:28.560983 9370 consensus_peers.cc:485] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de -> Peer e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72:35005): Couldn't send request to peer e01cf117ef5d446590c06dd1a6ab7ef7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5f84e791335948e180f5575f8174efd1. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:28.562981 13217 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.563345 9830 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.563549 9696 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.565008 11362 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.565701 9035 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.566843 8895 catalog_manager.cc:5484] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.567165 9168 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.568310 9301 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:28.579545 9564 consensus_queue.cc:237] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:28.581597 9830 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.581727 11362 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.581777 9301 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.582073 9696 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.582113 13230 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.582661 13230 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240502 14:01:28.582895 9501 consensus_peers.cc:485] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca -> Peer 304274b88d234198a3892e38c05014a5 (127.24.66.73:42559): Couldn't send request to peer 304274b88d234198a3892e38c05014a5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 21d4acfb937f4dd4b992c338fc6b3237. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:28.583228 13029 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.583382 13231 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.584276 9168 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.584666 9035 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:28.585268 13230 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.585711 13029 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:28.585975 13232 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.586493 9696 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.586959 9168 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.588286 11362 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.588615 9301 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.589265 8888 catalog_manager.cc:5484] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca reported cstate change: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:28.590468 9830 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.594517 9035 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:28.811177 13388 ts_tablet_manager.cc:902] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5: Initiating tablet copy from peer d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375)
I20240502 14:01:28.811769 13388 tablet_copy_client.cc:308] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.69:36375
I20240502 14:01:28.812232 9586 tablet_copy_service.cc:140] P d0f45862d7f14ce1976c1a37842f25ca: Received BeginTabletCopySession request for tablet 7289a5be3b3f437f9a2770baa42b89a8 from peer 304274b88d234198a3892e38c05014a5 ({username='slave'} at 127.24.66.73:46083)
I20240502 14:01:28.812374 9586 tablet_copy_service.cc:161] P d0f45862d7f14ce1976c1a37842f25ca: Beginning new tablet copy session on tablet 7289a5be3b3f437f9a2770baa42b89a8 from peer 304274b88d234198a3892e38c05014a5 at {username='slave'} at 127.24.66.73:46083: session id = 304274b88d234198a3892e38c05014a5-7289a5be3b3f437f9a2770baa42b89a8
I20240502 14:01:28.813185 9586 tablet_copy_source_session.cc:205] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:28.814059 13388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7289a5be3b3f437f9a2770baa42b89a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:28.816078 13388 tablet_copy_client.cc:783] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:28.816223 13388 tablet_copy_client.cc:647] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:28.817343 13388 tablet_copy_client.cc:523] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:28.818459 13388 tablet_bootstrap.cc:492] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5: Bootstrap starting.
I20240502 14:01:28.820556 13393 ts_tablet_manager.cc:902] T ada4c2cc1749411d8f41f882b7c64b07 P 304274b88d234198a3892e38c05014a5: Initiating tablet copy from peer d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837)
I20240502 14:01:28.821085 13393 tablet_copy_client.cc:308] T ada4c2cc1749411d8f41f882b7c64b07 P 304274b88d234198a3892e38c05014a5: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.66:38837
I20240502 14:01:28.825606 9190 tablet_copy_service.cc:140] P d83af1bc823a4f2ca531f0d6825f8e16: Received BeginTabletCopySession request for tablet ada4c2cc1749411d8f41f882b7c64b07 from peer 304274b88d234198a3892e38c05014a5 ({username='slave'} at 127.24.66.73:52497)
I20240502 14:01:28.825724 9190 tablet_copy_service.cc:161] P d83af1bc823a4f2ca531f0d6825f8e16: Beginning new tablet copy session on tablet ada4c2cc1749411d8f41f882b7c64b07 from peer 304274b88d234198a3892e38c05014a5 at {username='slave'} at 127.24.66.73:52497: session id = 304274b88d234198a3892e38c05014a5-ada4c2cc1749411d8f41f882b7c64b07
I20240502 14:01:28.826604 9190 tablet_copy_source_session.cc:205] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:28.827489 13393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ada4c2cc1749411d8f41f882b7c64b07. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:28.829720 13393 tablet_copy_client.cc:783] T ada4c2cc1749411d8f41f882b7c64b07 P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:28.829872 13393 tablet_copy_client.cc:647] T ada4c2cc1749411d8f41f882b7c64b07 P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:28.831022 13393 tablet_copy_client.cc:523] T ada4c2cc1749411d8f41f882b7c64b07 P 304274b88d234198a3892e38c05014a5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:28.832302 13393 tablet_bootstrap.cc:492] T ada4c2cc1749411d8f41f882b7c64b07 P 304274b88d234198a3892e38c05014a5: Bootstrap starting.
I20240502 14:01:28.848385 13401 ts_tablet_manager.cc:902] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7: Initiating tablet copy from peer ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:28.849054 13401 tablet_copy_client.cc:308] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.67:34749
I20240502 14:01:28.849588 13388 tablet_bootstrap.cc:492] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:28.849732 9322 tablet_copy_service.cc:140] P ccfb5bbc1bb54793812d11170ac2d25f: Received BeginTabletCopySession request for tablet bfa2e6af0fd544fa8bfdaefedabb5e68 from peer e01cf117ef5d446590c06dd1a6ab7ef7 ({username='slave'} at 127.24.66.72:33161)
I20240502 14:01:28.849857 9322 tablet_copy_service.cc:161] P ccfb5bbc1bb54793812d11170ac2d25f: Beginning new tablet copy session on tablet bfa2e6af0fd544fa8bfdaefedabb5e68 from peer e01cf117ef5d446590c06dd1a6ab7ef7 at {username='slave'} at 127.24.66.72:33161: session id = e01cf117ef5d446590c06dd1a6ab7ef7-bfa2e6af0fd544fa8bfdaefedabb5e68
I20240502 14:01:28.849905 13388 tablet_bootstrap.cc:492] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5: Bootstrap complete.
I20240502 14:01:28.850086 13388 ts_tablet_manager.cc:1367] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5: Time spent bootstrapping tablet: real 0.032s user 0.024s sys 0.005s
I20240502 14:01:28.850567 13388 raft_consensus.cc:348] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:28.850713 13388 raft_consensus.cc:729] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 304274b88d234198a3892e38c05014a5, State: Initialized, Role: LEARNER
I20240502 14:01:28.850845 9322 tablet_copy_source_session.cc:205] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:28.850822 13388 consensus_queue.cc:260] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:28.851212 13388 ts_tablet_manager.cc:1398] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:28.851812 9586 tablet_copy_service.cc:342] P d0f45862d7f14ce1976c1a37842f25ca: Request end of tablet copy session 304274b88d234198a3892e38c05014a5-7289a5be3b3f437f9a2770baa42b89a8 received from {username='slave'} at 127.24.66.73:46083
I20240502 14:01:28.851912 9586 tablet_copy_service.cc:433] P d0f45862d7f14ce1976c1a37842f25ca: ending tablet copy session 304274b88d234198a3892e38c05014a5-7289a5be3b3f437f9a2770baa42b89a8 on tablet 7289a5be3b3f437f9a2770baa42b89a8 with peer 304274b88d234198a3892e38c05014a5
I20240502 14:01:28.852478 13401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfa2e6af0fd544fa8bfdaefedabb5e68. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:28.855237 13401 tablet_copy_client.cc:783] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:28.855437 13401 tablet_copy_client.cc:647] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:28.857002 13401 tablet_copy_client.cc:523] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:28.858181 13401 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap starting.
I20240502 14:01:28.862444 13393 tablet_bootstrap.cc:492] T ada4c2cc1749411d8f41f882b7c64b07 P 304274b88d234198a3892e38c05014a5: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:28.862737 13393 tablet_bootstrap.cc:492] T ada4c2cc1749411d8f41f882b7c64b07 P 304274b88d234198a3892e38c05014a5: Bootstrap complete.
I20240502 14:01:28.862907 13393 ts_tablet_manager.cc:1367] T ada4c2cc1749411d8f41f882b7c64b07 P 304274b88d234198a3892e38c05014a5: Time spent bootstrapping tablet: real 0.031s user 0.018s sys 0.007s
I20240502 14:01:28.863363 13393 raft_consensus.cc:348] T ada4c2cc1749411d8f41f882b7c64b07 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:28.863503 13393 raft_consensus.cc:729] T ada4c2cc1749411d8f41f882b7c64b07 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 304274b88d234198a3892e38c05014a5, State: Initialized, Role: LEARNER
I20240502 14:01:28.863602 13393 consensus_queue.cc:260] T ada4c2cc1749411d8f41f882b7c64b07 P 304274b88d234198a3892e38c05014a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:28.863898 13393 ts_tablet_manager.cc:1398] T ada4c2cc1749411d8f41f882b7c64b07 P 304274b88d234198a3892e38c05014a5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:28.865000 9190 tablet_copy_service.cc:342] P d83af1bc823a4f2ca531f0d6825f8e16: Request end of tablet copy session 304274b88d234198a3892e38c05014a5-ada4c2cc1749411d8f41f882b7c64b07 received from {username='slave'} at 127.24.66.73:52497
I20240502 14:01:28.865089 9190 tablet_copy_service.cc:433] P d83af1bc823a4f2ca531f0d6825f8e16: ending tablet copy session 304274b88d234198a3892e38c05014a5-ada4c2cc1749411d8f41f882b7c64b07 on tablet ada4c2cc1749411d8f41f882b7c64b07 with peer 304274b88d234198a3892e38c05014a5
I20240502 14:01:28.882450 13401 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:28.882769 13401 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap complete.
I20240502 14:01:28.882973 13401 ts_tablet_manager.cc:1367] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent bootstrapping tablet: real 0.025s user 0.023s sys 0.000s
I20240502 14:01:28.883450 13401 raft_consensus.cc:348] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:28.883708 13401 raft_consensus.cc:729] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e01cf117ef5d446590c06dd1a6ab7ef7, State: Initialized, Role: LEARNER
I20240502 14:01:28.883848 13401 consensus_queue.cc:260] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:28.884433 13401 ts_tablet_manager.cc:1398] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 14:01:28.887848 9322 tablet_copy_service.cc:342] P ccfb5bbc1bb54793812d11170ac2d25f: Request end of tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-bfa2e6af0fd544fa8bfdaefedabb5e68 received from {username='slave'} at 127.24.66.72:33161
I20240502 14:01:28.887935 9322 tablet_copy_service.cc:433] P ccfb5bbc1bb54793812d11170ac2d25f: ending tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-bfa2e6af0fd544fa8bfdaefedabb5e68 on tablet bfa2e6af0fd544fa8bfdaefedabb5e68 with peer e01cf117ef5d446590c06dd1a6ab7ef7
I20240502 14:01:28.960925 13463 ts_tablet_manager.cc:902] T 9eef89d7e129492bab305a1b5009aace P 535b19a469c34efaaf49ca1cc2e1e5fc: Initiating tablet copy from peer 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907)
I20240502 14:01:28.961493 13463 tablet_copy_client.cc:308] T 9eef89d7e129492bab305a1b5009aace P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:42907
I20240502 14:01:28.962051 9454 tablet_copy_service.cc:140] P 1a490f170c614ee4900ef16c3287c7de: Received BeginTabletCopySession request for tablet 9eef89d7e129492bab305a1b5009aace from peer 535b19a469c34efaaf49ca1cc2e1e5fc ({username='slave'} at 127.24.66.74:50277)
I20240502 14:01:28.962174 9454 tablet_copy_service.cc:161] P 1a490f170c614ee4900ef16c3287c7de: Beginning new tablet copy session on tablet 9eef89d7e129492bab305a1b5009aace from peer 535b19a469c34efaaf49ca1cc2e1e5fc at {username='slave'} at 127.24.66.74:50277: session id = 535b19a469c34efaaf49ca1cc2e1e5fc-9eef89d7e129492bab305a1b5009aace
I20240502 14:01:28.963052 9454 tablet_copy_source_session.cc:205] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:28.963927 13463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eef89d7e129492bab305a1b5009aace. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:28.966022 13463 tablet_copy_client.cc:783] T 9eef89d7e129492bab305a1b5009aace P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:28.966171 13463 tablet_copy_client.cc:647] T 9eef89d7e129492bab305a1b5009aace P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:28.967842 13463 tablet_copy_client.cc:523] T 9eef89d7e129492bab305a1b5009aace P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:28.969635 13468 ts_tablet_manager.cc:902] T 39bef244549d4bb7a980bf7bc52a01b4 P 535b19a469c34efaaf49ca1cc2e1e5fc: Initiating tablet copy from peer 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:28.969668 13463 tablet_bootstrap.cc:492] T 9eef89d7e129492bab305a1b5009aace P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap starting.
I20240502 14:01:28.970335 13468 tablet_copy_client.cc:308] T 39bef244549d4bb7a980bf7bc52a01b4 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.71:33679
I20240502 14:01:28.970881 9850 tablet_copy_service.cc:140] P 0488b0e4ed994db19d40aed8c15f945b: Received BeginTabletCopySession request for tablet 39bef244549d4bb7a980bf7bc52a01b4 from peer 535b19a469c34efaaf49ca1cc2e1e5fc ({username='slave'} at 127.24.66.74:53801)
I20240502 14:01:28.971026 9850 tablet_copy_service.cc:161] P 0488b0e4ed994db19d40aed8c15f945b: Beginning new tablet copy session on tablet 39bef244549d4bb7a980bf7bc52a01b4 from peer 535b19a469c34efaaf49ca1cc2e1e5fc at {username='slave'} at 127.24.66.74:53801: session id = 535b19a469c34efaaf49ca1cc2e1e5fc-39bef244549d4bb7a980bf7bc52a01b4
I20240502 14:01:28.972318 9850 tablet_copy_source_session.cc:205] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:28.973282 13468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39bef244549d4bb7a980bf7bc52a01b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:28.975991 13468 tablet_copy_client.cc:783] T 39bef244549d4bb7a980bf7bc52a01b4 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:28.976449 13468 tablet_copy_client.cc:647] T 39bef244549d4bb7a980bf7bc52a01b4 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:28.976871 13393 ts_tablet_manager.cc:902] T 21d4acfb937f4dd4b992c338fc6b3237 P 304274b88d234198a3892e38c05014a5: Initiating tablet copy from peer d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375)
I20240502 14:01:28.977335 13393 tablet_copy_client.cc:308] T 21d4acfb937f4dd4b992c338fc6b3237 P 304274b88d234198a3892e38c05014a5: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.69:36375
I20240502 14:01:28.978163 9586 tablet_copy_service.cc:140] P d0f45862d7f14ce1976c1a37842f25ca: Received BeginTabletCopySession request for tablet 21d4acfb937f4dd4b992c338fc6b3237 from peer 304274b88d234198a3892e38c05014a5 ({username='slave'} at 127.24.66.73:46083)
I20240502 14:01:28.978283 9586 tablet_copy_service.cc:161] P d0f45862d7f14ce1976c1a37842f25ca: Beginning new tablet copy session on tablet 21d4acfb937f4dd4b992c338fc6b3237 from peer 304274b88d234198a3892e38c05014a5 at {username='slave'} at 127.24.66.73:46083: session id = 304274b88d234198a3892e38c05014a5-21d4acfb937f4dd4b992c338fc6b3237
I20240502 14:01:28.979574 9586 tablet_copy_source_session.cc:205] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:28.979959 13468 tablet_copy_client.cc:523] T 39bef244549d4bb7a980bf7bc52a01b4 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:28.980662 13393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d4acfb937f4dd4b992c338fc6b3237. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:28.981562 13468 tablet_bootstrap.cc:492] T 39bef244549d4bb7a980bf7bc52a01b4 P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap starting.
I20240502 14:01:28.983117 13393 tablet_copy_client.cc:783] T 21d4acfb937f4dd4b992c338fc6b3237 P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:28.983256 13393 tablet_copy_client.cc:647] T 21d4acfb937f4dd4b992c338fc6b3237 P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:28.988988 13393 tablet_copy_client.cc:523] T 21d4acfb937f4dd4b992c338fc6b3237 P 304274b88d234198a3892e38c05014a5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:28.990711 13393 tablet_bootstrap.cc:492] T 21d4acfb937f4dd4b992c338fc6b3237 P 304274b88d234198a3892e38c05014a5: Bootstrap starting.
I20240502 14:01:28.994510 13463 tablet_bootstrap.cc:492] T 9eef89d7e129492bab305a1b5009aace P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:29.008875 13401 ts_tablet_manager.cc:902] T 97fa16722a1f4adcbf8a5dc3fdfab77a P e01cf117ef5d446590c06dd1a6ab7ef7: Initiating tablet copy from peer 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:29.009409 13401 tablet_copy_client.cc:308] T 97fa16722a1f4adcbf8a5dc3fdfab77a P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.71:33679
I20240502 14:01:29.016579 13463 tablet_bootstrap.cc:492] T 9eef89d7e129492bab305a1b5009aace P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap complete.
I20240502 14:01:29.016824 13463 ts_tablet_manager.cc:1367] T 9eef89d7e129492bab305a1b5009aace P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent bootstrapping tablet: real 0.047s user 0.025s sys 0.000s
I20240502 14:01:29.017367 13463 raft_consensus.cc:348] T 9eef89d7e129492bab305a1b5009aace P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:29.017529 13463 raft_consensus.cc:729] T 9eef89d7e129492bab305a1b5009aace P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 535b19a469c34efaaf49ca1cc2e1e5fc, State: Initialized, Role: LEARNER
I20240502 14:01:29.017654 13463 consensus_queue.cc:260] T 9eef89d7e129492bab305a1b5009aace P 535b19a469c34efaaf49ca1cc2e1e5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:29.018010 13463 ts_tablet_manager.cc:1398] T 9eef89d7e129492bab305a1b5009aace P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:29.019546 9454 tablet_copy_service.cc:342] P 1a490f170c614ee4900ef16c3287c7de: Request end of tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-9eef89d7e129492bab305a1b5009aace received from {username='slave'} at 127.24.66.74:50277
I20240502 14:01:29.019649 9454 tablet_copy_service.cc:433] P 1a490f170c614ee4900ef16c3287c7de: ending tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-9eef89d7e129492bab305a1b5009aace on tablet 9eef89d7e129492bab305a1b5009aace with peer 535b19a469c34efaaf49ca1cc2e1e5fc
I20240502 14:01:29.021915 9850 tablet_copy_service.cc:140] P 0488b0e4ed994db19d40aed8c15f945b: Received BeginTabletCopySession request for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a from peer e01cf117ef5d446590c06dd1a6ab7ef7 ({username='slave'} at 127.24.66.72:37729)
I20240502 14:01:29.022027 9850 tablet_copy_service.cc:161] P 0488b0e4ed994db19d40aed8c15f945b: Beginning new tablet copy session on tablet 97fa16722a1f4adcbf8a5dc3fdfab77a from peer e01cf117ef5d446590c06dd1a6ab7ef7 at {username='slave'} at 127.24.66.72:37729: session id = e01cf117ef5d446590c06dd1a6ab7ef7-97fa16722a1f4adcbf8a5dc3fdfab77a
I20240502 14:01:29.022763 9850 tablet_copy_source_session.cc:205] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:29.023686 13401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:29.025964 13401 tablet_copy_client.cc:783] T 97fa16722a1f4adcbf8a5dc3fdfab77a P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:29.026100 13401 tablet_copy_client.cc:647] T 97fa16722a1f4adcbf8a5dc3fdfab77a P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:29.027233 13468 tablet_bootstrap.cc:492] T 39bef244549d4bb7a980bf7bc52a01b4 P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:29.027554 13468 tablet_bootstrap.cc:492] T 39bef244549d4bb7a980bf7bc52a01b4 P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap complete.
I20240502 14:01:29.027740 13468 ts_tablet_manager.cc:1367] T 39bef244549d4bb7a980bf7bc52a01b4 P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent bootstrapping tablet: real 0.046s user 0.040s sys 0.000s
I20240502 14:01:29.028298 13468 raft_consensus.cc:348] T 39bef244549d4bb7a980bf7bc52a01b4 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:29.028508 13468 raft_consensus.cc:729] T 39bef244549d4bb7a980bf7bc52a01b4 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 535b19a469c34efaaf49ca1cc2e1e5fc, State: Initialized, Role: LEARNER
I20240502 14:01:29.028635 13468 consensus_queue.cc:260] T 39bef244549d4bb7a980bf7bc52a01b4 P 535b19a469c34efaaf49ca1cc2e1e5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:29.028973 13468 ts_tablet_manager.cc:1398] T 39bef244549d4bb7a980bf7bc52a01b4 P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240502 14:01:29.029666 9850 tablet_copy_service.cc:342] P 0488b0e4ed994db19d40aed8c15f945b: Request end of tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-39bef244549d4bb7a980bf7bc52a01b4 received from {username='slave'} at 127.24.66.74:53801
I20240502 14:01:29.029743 9850 tablet_copy_service.cc:433] P 0488b0e4ed994db19d40aed8c15f945b: ending tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-39bef244549d4bb7a980bf7bc52a01b4 on tablet 39bef244549d4bb7a980bf7bc52a01b4 with peer 535b19a469c34efaaf49ca1cc2e1e5fc
I20240502 14:01:29.031556 13388 ts_tablet_manager.cc:902] T b1548ca1bcd24d288d30117baea53c12 P 304274b88d234198a3892e38c05014a5: Initiating tablet copy from peer 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907)
I20240502 14:01:29.032187 13388 tablet_copy_client.cc:308] T b1548ca1bcd24d288d30117baea53c12 P 304274b88d234198a3892e38c05014a5: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:42907
I20240502 14:01:29.032990 9454 tablet_copy_service.cc:140] P 1a490f170c614ee4900ef16c3287c7de: Received BeginTabletCopySession request for tablet b1548ca1bcd24d288d30117baea53c12 from peer 304274b88d234198a3892e38c05014a5 ({username='slave'} at 127.24.66.73:36691)
I20240502 14:01:29.033115 9454 tablet_copy_service.cc:161] P 1a490f170c614ee4900ef16c3287c7de: Beginning new tablet copy session on tablet b1548ca1bcd24d288d30117baea53c12 from peer 304274b88d234198a3892e38c05014a5 at {username='slave'} at 127.24.66.73:36691: session id = 304274b88d234198a3892e38c05014a5-b1548ca1bcd24d288d30117baea53c12
I20240502 14:01:29.034404 9454 tablet_copy_source_session.cc:205] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:29.035321 13401 tablet_copy_client.cc:523] T 97fa16722a1f4adcbf8a5dc3fdfab77a P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:29.036756 13401 tablet_bootstrap.cc:492] T 97fa16722a1f4adcbf8a5dc3fdfab77a P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap starting.
I20240502 14:01:29.037659 13393 tablet_bootstrap.cc:492] T 21d4acfb937f4dd4b992c338fc6b3237 P 304274b88d234198a3892e38c05014a5: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:29.037917 13393 tablet_bootstrap.cc:492] T 21d4acfb937f4dd4b992c338fc6b3237 P 304274b88d234198a3892e38c05014a5: Bootstrap complete.
I20240502 14:01:29.038106 13393 ts_tablet_manager.cc:1367] T 21d4acfb937f4dd4b992c338fc6b3237 P 304274b88d234198a3892e38c05014a5: Time spent bootstrapping tablet: real 0.047s user 0.029s sys 0.000s
I20240502 14:01:29.038522 13388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1548ca1bcd24d288d30117baea53c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:29.038594 13393 raft_consensus.cc:348] T 21d4acfb937f4dd4b992c338fc6b3237 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:29.038733 13393 raft_consensus.cc:729] T 21d4acfb937f4dd4b992c338fc6b3237 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 304274b88d234198a3892e38c05014a5, State: Initialized, Role: LEARNER
I20240502 14:01:29.038841 13393 consensus_queue.cc:260] T 21d4acfb937f4dd4b992c338fc6b3237 P 304274b88d234198a3892e38c05014a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:29.039669 13393 ts_tablet_manager.cc:1398] T 21d4acfb937f4dd4b992c338fc6b3237 P 304274b88d234198a3892e38c05014a5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240502 14:01:29.041054 13388 tablet_copy_client.cc:783] T b1548ca1bcd24d288d30117baea53c12 P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:29.041508 9586 tablet_copy_service.cc:342] P d0f45862d7f14ce1976c1a37842f25ca: Request end of tablet copy session 304274b88d234198a3892e38c05014a5-21d4acfb937f4dd4b992c338fc6b3237 received from {username='slave'} at 127.24.66.73:46083
I20240502 14:01:29.041592 9586 tablet_copy_service.cc:433] P d0f45862d7f14ce1976c1a37842f25ca: ending tablet copy session 304274b88d234198a3892e38c05014a5-21d4acfb937f4dd4b992c338fc6b3237 on tablet 21d4acfb937f4dd4b992c338fc6b3237 with peer 304274b88d234198a3892e38c05014a5
I20240502 14:01:29.043350 13388 tablet_copy_client.cc:647] T b1548ca1bcd24d288d30117baea53c12 P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:29.047271 13388 tablet_copy_client.cc:523] T b1548ca1bcd24d288d30117baea53c12 P 304274b88d234198a3892e38c05014a5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:29.048838 13388 tablet_bootstrap.cc:492] T b1548ca1bcd24d288d30117baea53c12 P 304274b88d234198a3892e38c05014a5: Bootstrap starting.
I20240502 14:01:29.073309 13401 tablet_bootstrap.cc:492] T 97fa16722a1f4adcbf8a5dc3fdfab77a P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:29.073643 13401 tablet_bootstrap.cc:492] T 97fa16722a1f4adcbf8a5dc3fdfab77a P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap complete.
I20240502 14:01:29.073848 13401 ts_tablet_manager.cc:1367] T 97fa16722a1f4adcbf8a5dc3fdfab77a P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent bootstrapping tablet: real 0.037s user 0.023s sys 0.000s
I20240502 14:01:29.074312 13401 raft_consensus.cc:348] T 97fa16722a1f4adcbf8a5dc3fdfab77a P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:29.074465 13401 raft_consensus.cc:729] T 97fa16722a1f4adcbf8a5dc3fdfab77a P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e01cf117ef5d446590c06dd1a6ab7ef7, State: Initialized, Role: LEARNER
I20240502 14:01:29.074576 13401 consensus_queue.cc:260] T 97fa16722a1f4adcbf8a5dc3fdfab77a P e01cf117ef5d446590c06dd1a6ab7ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:29.074894 13401 ts_tablet_manager.cc:1398] T 97fa16722a1f4adcbf8a5dc3fdfab77a P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:29.075472 13393 ts_tablet_manager.cc:902] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5: Initiating tablet copy from peer 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907)
I20240502 14:01:29.075439 9850 tablet_copy_service.cc:342] P 0488b0e4ed994db19d40aed8c15f945b: Request end of tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-97fa16722a1f4adcbf8a5dc3fdfab77a received from {username='slave'} at 127.24.66.72:37729
I20240502 14:01:29.075548 9850 tablet_copy_service.cc:433] P 0488b0e4ed994db19d40aed8c15f945b: ending tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-97fa16722a1f4adcbf8a5dc3fdfab77a on tablet 97fa16722a1f4adcbf8a5dc3fdfab77a with peer e01cf117ef5d446590c06dd1a6ab7ef7
I20240502 14:01:29.076084 13393 tablet_copy_client.cc:308] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:42907
I20240502 14:01:29.076608 9454 tablet_copy_service.cc:140] P 1a490f170c614ee4900ef16c3287c7de: Received BeginTabletCopySession request for tablet 8c10fb9748c747c6a89066fa113cda3c from peer 304274b88d234198a3892e38c05014a5 ({username='slave'} at 127.24.66.73:36691)
I20240502 14:01:29.076706 9454 tablet_copy_service.cc:161] P 1a490f170c614ee4900ef16c3287c7de: Beginning new tablet copy session on tablet 8c10fb9748c747c6a89066fa113cda3c from peer 304274b88d234198a3892e38c05014a5 at {username='slave'} at 127.24.66.73:36691: session id = 304274b88d234198a3892e38c05014a5-8c10fb9748c747c6a89066fa113cda3c
I20240502 14:01:29.078053 9454 tablet_copy_source_session.cc:205] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:29.080091 13393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c10fb9748c747c6a89066fa113cda3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:29.083299 13393 tablet_copy_client.cc:783] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:29.083496 13393 tablet_copy_client.cc:647] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:29.084786 13393 tablet_copy_client.cc:523] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:29.086943 13393 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5: Bootstrap starting.
I20240502 14:01:29.098563 13388 tablet_bootstrap.cc:492] T b1548ca1bcd24d288d30117baea53c12 P 304274b88d234198a3892e38c05014a5: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:29.098861 13388 tablet_bootstrap.cc:492] T b1548ca1bcd24d288d30117baea53c12 P 304274b88d234198a3892e38c05014a5: Bootstrap complete.
I20240502 14:01:29.099030 13388 ts_tablet_manager.cc:1367] T b1548ca1bcd24d288d30117baea53c12 P 304274b88d234198a3892e38c05014a5: Time spent bootstrapping tablet: real 0.050s user 0.020s sys 0.000s
I20240502 14:01:29.099440 13388 raft_consensus.cc:348] T b1548ca1bcd24d288d30117baea53c12 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:29.099560 13388 raft_consensus.cc:729] T b1548ca1bcd24d288d30117baea53c12 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 304274b88d234198a3892e38c05014a5, State: Initialized, Role: LEARNER
I20240502 14:01:29.099668 13388 consensus_queue.cc:260] T b1548ca1bcd24d288d30117baea53c12 P 304274b88d234198a3892e38c05014a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:29.100039 13388 ts_tablet_manager.cc:1398] T b1548ca1bcd24d288d30117baea53c12 P 304274b88d234198a3892e38c05014a5: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 14:01:29.100821 9454 tablet_copy_service.cc:342] P 1a490f170c614ee4900ef16c3287c7de: Request end of tablet copy session 304274b88d234198a3892e38c05014a5-b1548ca1bcd24d288d30117baea53c12 received from {username='slave'} at 127.24.66.73:36691
I20240502 14:01:29.100946 9454 tablet_copy_service.cc:433] P 1a490f170c614ee4900ef16c3287c7de: ending tablet copy session 304274b88d234198a3892e38c05014a5-b1548ca1bcd24d288d30117baea53c12 on tablet b1548ca1bcd24d288d30117baea53c12 with peer 304274b88d234198a3892e38c05014a5
I20240502 14:01:29.111735 13401 ts_tablet_manager.cc:902] T 6232f47837664d7486612ba1d291f2b8 P e01cf117ef5d446590c06dd1a6ab7ef7: Initiating tablet copy from peer 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:29.112241 13401 tablet_copy_client.cc:308] T 6232f47837664d7486612ba1d291f2b8 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.71:33679
I20240502 14:01:29.112835 9850 tablet_copy_service.cc:140] P 0488b0e4ed994db19d40aed8c15f945b: Received BeginTabletCopySession request for tablet 6232f47837664d7486612ba1d291f2b8 from peer e01cf117ef5d446590c06dd1a6ab7ef7 ({username='slave'} at 127.24.66.72:37729)
I20240502 14:01:29.112955 9850 tablet_copy_service.cc:161] P 0488b0e4ed994db19d40aed8c15f945b: Beginning new tablet copy session on tablet 6232f47837664d7486612ba1d291f2b8 from peer e01cf117ef5d446590c06dd1a6ab7ef7 at {username='slave'} at 127.24.66.72:37729: session id = e01cf117ef5d446590c06dd1a6ab7ef7-6232f47837664d7486612ba1d291f2b8
I20240502 14:01:29.113762 9850 tablet_copy_source_session.cc:205] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:29.114627 13401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6232f47837664d7486612ba1d291f2b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:29.116835 13401 tablet_copy_client.cc:783] T 6232f47837664d7486612ba1d291f2b8 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:29.116998 13401 tablet_copy_client.cc:647] T 6232f47837664d7486612ba1d291f2b8 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:29.119330 13401 tablet_copy_client.cc:523] T 6232f47837664d7486612ba1d291f2b8 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:29.120422 13401 tablet_bootstrap.cc:492] T 6232f47837664d7486612ba1d291f2b8 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap starting.
I20240502 14:01:29.136129 13501 ts_tablet_manager.cc:902] T 5f84e791335948e180f5575f8174efd1 P e01cf117ef5d446590c06dd1a6ab7ef7: Initiating tablet copy from peer 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907)
I20240502 14:01:29.136821 13501 tablet_copy_client.cc:308] T 5f84e791335948e180f5575f8174efd1 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:42907
I20240502 14:01:29.137428 9454 tablet_copy_service.cc:140] P 1a490f170c614ee4900ef16c3287c7de: Received BeginTabletCopySession request for tablet 5f84e791335948e180f5575f8174efd1 from peer e01cf117ef5d446590c06dd1a6ab7ef7 ({username='slave'} at 127.24.66.72:40361)
I20240502 14:01:29.137542 9454 tablet_copy_service.cc:161] P 1a490f170c614ee4900ef16c3287c7de: Beginning new tablet copy session on tablet 5f84e791335948e180f5575f8174efd1 from peer e01cf117ef5d446590c06dd1a6ab7ef7 at {username='slave'} at 127.24.66.72:40361: session id = e01cf117ef5d446590c06dd1a6ab7ef7-5f84e791335948e180f5575f8174efd1
I20240502 14:01:29.138527 9454 tablet_copy_source_session.cc:205] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:29.139293 13393 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:29.139647 13393 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5: Bootstrap complete.
I20240502 14:01:29.139647 13501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f84e791335948e180f5575f8174efd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:29.139858 13393 ts_tablet_manager.cc:1367] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5: Time spent bootstrapping tablet: real 0.053s user 0.045s sys 0.000s
I20240502 14:01:29.140561 13393 raft_consensus.cc:348] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:29.140764 13393 raft_consensus.cc:729] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 304274b88d234198a3892e38c05014a5, State: Initialized, Role: LEARNER
I20240502 14:01:29.140900 13393 consensus_queue.cc:260] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:29.141319 13393 ts_tablet_manager.cc:1398] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 14:01:29.141922 9454 tablet_copy_service.cc:342] P 1a490f170c614ee4900ef16c3287c7de: Request end of tablet copy session 304274b88d234198a3892e38c05014a5-8c10fb9748c747c6a89066fa113cda3c received from {username='slave'} at 127.24.66.73:36691
I20240502 14:01:29.142012 9454 tablet_copy_service.cc:433] P 1a490f170c614ee4900ef16c3287c7de: ending tablet copy session 304274b88d234198a3892e38c05014a5-8c10fb9748c747c6a89066fa113cda3c on tablet 8c10fb9748c747c6a89066fa113cda3c with peer 304274b88d234198a3892e38c05014a5
I20240502 14:01:29.144281 13401 tablet_bootstrap.cc:492] T 6232f47837664d7486612ba1d291f2b8 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:29.146392 13501 tablet_copy_client.cc:783] T 5f84e791335948e180f5575f8174efd1 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:29.146564 13501 tablet_copy_client.cc:647] T 5f84e791335948e180f5575f8174efd1 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:29.147930 13501 tablet_copy_client.cc:523] T 5f84e791335948e180f5575f8174efd1 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:29.150346 13501 tablet_bootstrap.cc:492] T 5f84e791335948e180f5575f8174efd1 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap starting.
I20240502 14:01:29.172575 13401 tablet_bootstrap.cc:492] T 6232f47837664d7486612ba1d291f2b8 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap complete.
I20240502 14:01:29.172834 13401 ts_tablet_manager.cc:1367] T 6232f47837664d7486612ba1d291f2b8 P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent bootstrapping tablet: real 0.052s user 0.023s sys 0.001s
I20240502 14:01:29.173394 13401 raft_consensus.cc:348] T 6232f47837664d7486612ba1d291f2b8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:29.173566 13401 raft_consensus.cc:729] T 6232f47837664d7486612ba1d291f2b8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e01cf117ef5d446590c06dd1a6ab7ef7, State: Initialized, Role: LEARNER
I20240502 14:01:29.173702 13401 consensus_queue.cc:260] T 6232f47837664d7486612ba1d291f2b8 P e01cf117ef5d446590c06dd1a6ab7ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:29.174046 13401 ts_tablet_manager.cc:1398] T 6232f47837664d7486612ba1d291f2b8 P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:29.177454 9850 tablet_copy_service.cc:342] P 0488b0e4ed994db19d40aed8c15f945b: Request end of tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-6232f47837664d7486612ba1d291f2b8 received from {username='slave'} at 127.24.66.72:37729
I20240502 14:01:29.177532 9850 tablet_copy_service.cc:433] P 0488b0e4ed994db19d40aed8c15f945b: ending tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-6232f47837664d7486612ba1d291f2b8 on tablet 6232f47837664d7486612ba1d291f2b8 with peer e01cf117ef5d446590c06dd1a6ab7ef7
I20240502 14:01:29.198092 13501 tablet_bootstrap.cc:492] T 5f84e791335948e180f5575f8174efd1 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:29.198393 13501 tablet_bootstrap.cc:492] T 5f84e791335948e180f5575f8174efd1 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap complete.
I20240502 14:01:29.198592 13501 ts_tablet_manager.cc:1367] T 5f84e791335948e180f5575f8174efd1 P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent bootstrapping tablet: real 0.048s user 0.024s sys 0.000s
I20240502 14:01:29.199194 13501 raft_consensus.cc:348] T 5f84e791335948e180f5575f8174efd1 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:29.199366 13501 raft_consensus.cc:729] T 5f84e791335948e180f5575f8174efd1 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e01cf117ef5d446590c06dd1a6ab7ef7, State: Initialized, Role: LEARNER
I20240502 14:01:29.199493 13501 consensus_queue.cc:260] T 5f84e791335948e180f5575f8174efd1 P e01cf117ef5d446590c06dd1a6ab7ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:29.199832 13501 ts_tablet_manager.cc:1398] T 5f84e791335948e180f5575f8174efd1 P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:29.200955 9454 tablet_copy_service.cc:342] P 1a490f170c614ee4900ef16c3287c7de: Request end of tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-5f84e791335948e180f5575f8174efd1 received from {username='slave'} at 127.24.66.72:40361
I20240502 14:01:29.201049 9454 tablet_copy_service.cc:433] P 1a490f170c614ee4900ef16c3287c7de: ending tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-5f84e791335948e180f5575f8174efd1 on tablet 5f84e791335948e180f5575f8174efd1 with peer e01cf117ef5d446590c06dd1a6ab7ef7
I20240502 14:01:29.270876 11099 raft_consensus.cc:1203] T ada4c2cc1749411d8f41f882b7c64b07 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 14:01:29.282554 11230 raft_consensus.cc:1203] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 14:01:29.321861 11099 raft_consensus.cc:1203] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 14:01:29.354523 11362 raft_consensus.cc:1203] T 9eef89d7e129492bab305a1b5009aace P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 14:01:29.409588 11362 raft_consensus.cc:1203] T 39bef244549d4bb7a980bf7bc52a01b4 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 14:01:29.459306 11099 raft_consensus.cc:1203] T 21d4acfb937f4dd4b992c338fc6b3237 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 14:01:29.547348 11099 raft_consensus.cc:1203] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 14:01:29.561666 11099 raft_consensus.cc:1203] T b1548ca1bcd24d288d30117baea53c12 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 14:01:29.570211 11230 raft_consensus.cc:1203] T 97fa16722a1f4adcbf8a5dc3fdfab77a P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 14:01:29.572559 11230 raft_consensus.cc:1203] T 6232f47837664d7486612ba1d291f2b8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 14:01:29.625188 11230 raft_consensus.cc:1203] T 5f84e791335948e180f5575f8174efd1 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 14:01:29.675930 13427 raft_consensus.cc:1050] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16: attempting to promote NON_VOTER 304274b88d234198a3892e38c05014a5 to VOTER
I20240502 14:01:29.676503 13427 consensus_queue.cc:237] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:29.678580 9564 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.678617 11230 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.678655 9302 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.679018 9830 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.679019 13016 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.679185 9696 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.679553 11099 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.679636 13016 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.679920 13620 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.680251 13620 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.680461 13621 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.680517 9036 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.680792 13620 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.682513 13620 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.683238 13016 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:29.684804 9696 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:29.685597 9302 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:29.685592 9830 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:29.686398 9564 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:29.688850 8892 catalog_manager.cc:5484] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:29.689203 11231 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:29.691112 9035 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:29.691416 11099 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:29.697497 9168 consensus_queue.cc:237] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:29.699088 9696 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:29.699108 11231 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:29.699172 9830 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:29.699504 13016 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:29.699672 13179 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:29.700044 11099 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:29.700022 13623 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:29.700568 9564 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:29.700752 13016 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:29.700855 9302 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:29.701310 13016 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:29.701529 13427 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:29.703558 13427 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:29.704371 11231 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:29.704716 9830 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:29.706703 11099 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:29.706722 9302 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:29.707412 8893 catalog_manager.cc:5484] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7 reported cstate change: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:29.707722 8882 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ada4c2cc1749411d8f41f882b7c64b07 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:29.708200 9697 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:29.709432 9564 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:29.713779 9018 tablet_service.cc:1514] Processing DeleteTablet for tablet ada4c2cc1749411d8f41f882b7c64b07 with delete_type TABLET_DATA_TOMBSTONED (TS bc9e4f2f668d48ea87d5aad4a2fb569d not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:45946
I20240502 14:01:29.714354 13636 tablet_replica.cc:330] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d: stopping tablet replica
I20240502 14:01:29.714670 13636 raft_consensus.cc:2229] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:29.715457 13636 raft_consensus.cc:2258] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:29.716603 13636 ts_tablet_manager.cc:1875] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:29.718693 13636 ts_tablet_manager.cc:1888] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 14:01:29.718775 13636 log.cc:1196] T ada4c2cc1749411d8f41f882b7c64b07 P bc9e4f2f668d48ea87d5aad4a2fb569d: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/wals/ada4c2cc1749411d8f41f882b7c64b07
I20240502 14:01:29.719343 8882 catalog_manager.cc:4830] TS bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459): tablet ada4c2cc1749411d8f41f882b7c64b07 (table rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]) successfully deleted
I20240502 14:01:29.806712 13054 raft_consensus.cc:1050] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de: attempting to promote NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc to VOTER
I20240502 14:01:29.807300 13054 consensus_queue.cc:237] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:29.809593 11362 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.811471 9830 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.811576 13701 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.811892 9302 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.811892 11099 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.811897 13700 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.812979 9697 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.812985 9168 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.813386 9035 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.813408 13699 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.813654 13698 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.813808 13700 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.813939 13215 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.814214 13699 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.816970 13698 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.817382 13562 raft_consensus.cc:1050] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f: attempting to promote NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 to VOTER
I20240502 14:01:29.817940 13562 consensus_queue.cc:237] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } }
I20240502 14:01:29.817942 11362 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.819545 9830 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.821313 11100 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.821372 9697 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.821653 9828 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.821653 8895 catalog_manager.cc:5484] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de reported cstate change: config changed from index 8 to 9, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:29.822737 9035 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.823040 9036 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.824224 9170 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.824568 9564 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.826099 11231 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.826064 9168 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.827553 13198 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.827970 13198 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.828320 13198 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.828677 13198 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.828974 13198 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.829309 13198 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.831574 11100 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.833379 9434 consensus_queue.cc:237] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:29.833506 13563 raft_consensus.cc:1050] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b: attempting to promote NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc to VOTER
I20240502 14:01:29.834054 13563 consensus_queue.cc:237] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:29.835330 9170 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:29.835714 13215 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:29.835781 9696 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:29.836668 13215 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:29.837818 9828 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:29.837816 11362 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.838663 9696 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.838881 12666 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:29.839104 11362 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:29.839594 9035 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.839660 9170 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.840098 12666 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:29.840268 11098 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.840349 9301 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.841572 9697 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.841598 11100 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:29.841606 9035 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:29.841953 12666 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:29.842160 13700 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:29.843609 13068 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.843700 13198 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.844152 13068 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.844272 13700 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.844478 13068 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.844764 13068 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.845016 13068 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.846498 13247 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.846530 8882 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 9eef89d7e129492bab305a1b5009aace with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:29.846680 9035 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.847631 11361 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.848709 9696 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.849406 9169 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.850293 8895 catalog_manager.cc:5484] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de reported cstate change: config changed from index 9 to 10, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:29.852365 9828 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.854437 9282 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eef89d7e129492bab305a1b5009aace with delete_type TABLET_DATA_TOMBSTONED (TS ccfb5bbc1bb54793812d11170ac2d25f not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:54252
I20240502 14:01:29.855576 13745 tablet_replica.cc:330] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f: stopping tablet replica
I20240502 14:01:29.856004 13745 raft_consensus.cc:2229] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:29.856515 13745 pending_rounds.cc:62] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f: Trying to abort 1 pending ops.
I20240502 14:01:29.856771 9564 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.856631 13745 pending_rounds.cc:68] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f: Aborting op as it isn't in flight: id { term: 1 index: 9 } timestamp: 7023241174249299968 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "9eef89d7e129492bab305a1b5009aace" old_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } } new_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } } }
I20240502 14:01:29.857050 13745 raft_consensus.cc:2894] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Aborting config change with OpId 1.9: Aborted: Op aborted
I20240502 14:01:29.857131 13745 raft_consensus.cc:2866] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20240502 14:01:29.857194 13745 raft_consensus.cc:2258] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:29.857612 11100 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.860044 13731 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20240502 14:01:29.861507 13712 consensus_queue.cc:1159] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" status { last_received { term: 1 index: 9 } last_committed_idx: 8 last_received_current_leader { term: 1 index: 9 } }
I20240502 14:01:29.862635 13745 ts_tablet_manager.cc:1875] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:29.865432 13745 ts_tablet_manager.cc:1888] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 14:01:29.865839 13745 log.cc:1196] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/wals/9eef89d7e129492bab305a1b5009aace
I20240502 14:01:29.866295 13732 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.868546 9695 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.869006 13717 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:29.869911 11099 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:29.872792 9828 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:29.872892 11098 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.875190 9300 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.875545 11362 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.876623 8889 catalog_manager.cc:5484] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:29.877264 8880 catalog_manager.cc:4830] TS ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749): tablet 9eef89d7e129492bab305a1b5009aace (table rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]) successfully deleted
I20240502 14:01:29.878182 8896 catalog_manager.cc:5484] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288 reported cstate change: config changed from index 8 to 9, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.878971 9564 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:29.879648 9036 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.880151 9168 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:29.881415 11231 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:29.882354 9035 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:29.886547 9697 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:29.887387 9168 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.890172 9300 consensus_queue.cc:237] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } }
I20240502 14:01:29.890753 9564 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.892371 9566 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:29.892632 9828 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:29.893482 13718 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:29.893836 13198 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:29.894668 11099 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:29.894824 9035 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:29.895308 13562 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:29.895375 11231 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:29.895529 13715 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:29.895768 13720 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:29.895777 9830 consensus_queue.cc:237] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:29.898056 11362 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:29.898243 9697 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:29.898545 9564 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:29.898715 13563 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:29.898958 13719 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:29.898243 9695 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:29.899387 13728 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:29.899924 13728 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:29.900462 9036 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:29.901118 13728 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:29.901307 11098 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:29.902236 13728 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:29.902477 13562 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:29.902796 11099 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:29.903074 9036 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:29.904546 9168 raft_consensus.cc:1261] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:29.905030 13728 consensus_queue.cc:1035] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:29.905589 8880 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet bfa2e6af0fd544fa8bfdaefedabb5e68 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:29.905478 9828 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:29.906306 11231 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:29.907837 8895 catalog_manager.cc:5484] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d reported cstate change: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:29.911304 9150 tablet_service.cc:1514] Processing DeleteTablet for tablet bfa2e6af0fd544fa8bfdaefedabb5e68 with delete_type TABLET_DATA_TOMBSTONED (TS d83af1bc823a4f2ca531f0d6825f8e16 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:37350
I20240502 14:01:29.912034 13728 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.913165 11098 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.914808 9565 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:29.914923 9566 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.916788 13747 tablet_replica.cc:330] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16: stopping tablet replica
I20240502 14:01:29.917690 9695 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:29.918471 9697 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.918594 8889 catalog_manager.cc:5484] T 39bef244549d4bb7a980bf7bc52a01b4 P 304274b88d234198a3892e38c05014a5 reported cstate change: config changed from index 9 to 10, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.920013 13747 raft_consensus.cc:2229] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:29.920451 13747 raft_consensus.cc:2258] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:29.922149 9282 tablet_service.cc:1514] Processing DeleteTablet for tablet 39bef244549d4bb7a980bf7bc52a01b4 with delete_type TABLET_DATA_TOMBSTONED (TS ccfb5bbc1bb54793812d11170ac2d25f not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:54252
I20240502 14:01:29.922314 8880 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 39bef244549d4bb7a980bf7bc52a01b4 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:29.922721 9035 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.922878 9170 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.922132 11362 raft_consensus.cc:2932] T 39bef244549d4bb7a980bf7bc52a01b4 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:29.923919 13745 tablet_replica.cc:330] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f: stopping tablet replica
I20240502 14:01:29.924070 13745 raft_consensus.cc:2229] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:29.924394 13745 raft_consensus.cc:2258] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:29.925350 13745 ts_tablet_manager.cc:1875] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:29.929728 13745 ts_tablet_manager.cc:1888] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 14:01:29.929814 13745 log.cc:1196] T 39bef244549d4bb7a980bf7bc52a01b4 P ccfb5bbc1bb54793812d11170ac2d25f: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/wals/39bef244549d4bb7a980bf7bc52a01b4
I20240502 14:01:29.930927 8880 catalog_manager.cc:4830] TS ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749): tablet 39bef244549d4bb7a980bf7bc52a01b4 (table rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]) successfully deleted
I20240502 14:01:29.932087 13747 ts_tablet_manager.cc:1875] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:29.933873 13230 raft_consensus.cc:1050] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca: attempting to promote NON_VOTER 304274b88d234198a3892e38c05014a5 to VOTER
I20240502 14:01:29.934458 13747 ts_tablet_manager.cc:1888] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 14:01:29.934548 13747 log.cc:1196] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d83af1bc823a4f2ca531f0d6825f8e16: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/wals/bfa2e6af0fd544fa8bfdaefedabb5e68
I20240502 14:01:29.934470 13230 consensus_queue.cc:237] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:29.935179 8882 catalog_manager.cc:4830] TS d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837): tablet bfa2e6af0fd544fa8bfdaefedabb5e68 (table rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]) successfully deleted
I20240502 14:01:29.936479 11231 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.936971 9828 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.937170 9170 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.937225 9035 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.937661 13233 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.937877 9697 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.937968 13762 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.938117 13233 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.938901 13759 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.939572 13230 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.939632 9300 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.940794 13758 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.943019 13758 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:29.943215 11098 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:29.943392 9828 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:29.943651 13762 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:29.945382 11098 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:29.945777 9035 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:29.946449 11231 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:29.947062 8896 catalog_manager.cc:5484] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:29.949998 9697 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:29.950377 9170 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:29.954279 9300 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.022981 12666 raft_consensus.cc:1050] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de: attempting to promote NON_VOTER 304274b88d234198a3892e38c05014a5 to VOTER
I20240502 14:01:30.023633 12666 consensus_queue.cc:237] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:30.025875 9300 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.025915 11231 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.026335 13215 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.026604 13699 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.027518 9170 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.028601 13699 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.029881 11098 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.029888 9035 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.030236 13701 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.030413 13700 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.030895 9828 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.031872 13700 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.033957 9697 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.034525 13701 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.035163 13698 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.035823 9828 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.037569 11098 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.040372 8896 catalog_manager.cc:5484] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 reported cstate change: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.040671 9170 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.042943 13054 raft_consensus.cc:1050] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de: attempting to promote NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 to VOTER
I20240502 14:01:30.043465 13054 consensus_queue.cc:237] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } }
I20240502 14:01:30.043798 11231 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.044236 9035 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.046463 11362 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.046502 9302 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.046784 9170 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.047283 9035 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.047336 13700 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.047613 13698 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.047669 11231 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.047760 13700 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.048877 9695 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.049706 9828 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.050176 13699 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.051844 13699 raft_consensus.cc:1050] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de: attempting to promote NON_VOTER 304274b88d234198a3892e38c05014a5 to VOTER
I20240502 14:01:30.052429 13699 consensus_queue.cc:237] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:30.054426 12666 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.054507 11098 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.054966 12666 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.055027 9695 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.055274 12666 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.055330 11231 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.055564 12666 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.056619 9170 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.056881 9830 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.057077 9300 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.057551 9035 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.057554 13701 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.057713 13700 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.058584 13731 raft_consensus.cc:1050] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b: attempting to promote NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 to VOTER
I20240502 14:01:30.059182 13731 consensus_queue.cc:237] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } }
I20240502 14:01:30.059445 9697 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.061461 9300 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.063402 9170 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.063411 9035 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.064028 13730 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.064415 13068 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.064721 11230 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.065107 9302 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.065696 11099 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.066381 13730 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.066713 13730 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.067500 13068 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.067636 9434 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.067925 13701 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.068315 13701 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.068944 13730 raft_consensus.cc:1050] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b: attempting to promote NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 to VOTER
I20240502 14:01:30.069442 13730 consensus_queue.cc:237] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } }
I20240502 14:01:30.071727 9301 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.071871 9036 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.072296 13563 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.072667 13728 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.073081 9168 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.073447 13840 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.073724 13728 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.074540 11231 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.075160 11100 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.076133 11362 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.076668 11231 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.076920 9169 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.077517 9434 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.078183 9433 consensus_queue.cc:237] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:30.078764 9034 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.080256 8890 catalog_manager.cc:5484] T 5f84e791335948e180f5575f8174efd1 P e01cf117ef5d446590c06dd1a6ab7ef7 reported cstate change: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.080830 9696 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.081436 13068 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.082171 13759 raft_consensus.cc:1050] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca: attempting to promote NON_VOTER 304274b88d234198a3892e38c05014a5 to VOTER
I20240502 14:01:30.082466 13700 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.082859 9301 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.082787 13759 consensus_queue.cc:237] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:30.083300 9170 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.083297 13698 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.084606 9695 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.084722 13840 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.084972 9697 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.085038 12666 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.085765 11231 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.086043 9830 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.086158 12666 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.086223 13728 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.086539 13728 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.087510 9828 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.087359 9696 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.087893 13233 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.088474 11100 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.088997 9300 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.089551 13215 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.089648 13233 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.089708 13563 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.089908 13730 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.090049 13563 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.092226 11100 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.092664 13233 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.093259 13699 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.093576 13699 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.093677 9434 consensus_queue.cc:237] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } }
I20240502 14:01:30.093564 9829 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.095554 11231 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.095566 9170 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.096071 11361 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.096012 13761 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.096312 13701 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.096501 12666 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.096859 9038 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.097348 13761 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.097910 9301 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.098129 9036 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.098507 13054 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.099383 13054 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.100422 12666 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.102687 11362 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.102555 13728 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.102684 11230 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.103008 8882 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 8c10fb9748c747c6a89066fa113cda3c with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:30.103422 9035 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.103440 11098 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.104432 11229 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.104941 9302 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.105473 8892 catalog_manager.cc:5484] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de reported cstate change: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:30.106998 13233 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.107385 11230 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.107905 8889 catalog_manager.cc:5484] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d reported cstate change: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.109148 9169 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.109402 9697 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:30.109994 13759 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:30.111057 9168 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.111518 13247 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.113313 11099 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.114084 9300 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.114151 13233 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.116910 9170 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.118561 8892 catalog_manager.cc:5484] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.118737 9018 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c10fb9748c747c6a89066fa113cda3c with delete_type TABLET_DATA_TOMBSTONED (TS bc9e4f2f668d48ea87d5aad4a2fb569d not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:45946
I20240502 14:01:30.118960 13636 tablet_replica.cc:330] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d: stopping tablet replica
I20240502 14:01:30.118860 9698 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.119096 13636 raft_consensus.cc:2229] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:30.119372 13636 raft_consensus.cc:2258] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:30.120483 9170 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.120785 9036 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.121598 11098 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.122252 9035 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.122560 9169 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.123075 9697 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.124955 9302 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.125200 9827 raft_consensus.cc:1261] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.125730 9430 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.125743 11360 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.126173 9431 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.127425 9697 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.128194 12666 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.128660 8889 catalog_manager.cc:5484] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d reported cstate change: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.130753 9697 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.131884 8887 catalog_manager.cc:5484] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc reported cstate change: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.132616 13701 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.133136 13701 consensus_queue.cc:1035] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.134339 13699 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.134716 9696 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.134768 11099 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.137493 11230 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.137843 9300 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.138444 9035 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.139303 9828 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.140594 8882 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 5f84e791335948e180f5575f8174efd1 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:30.142356 9302 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.143146 9697 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.146000 8896 catalog_manager.cc:5484] T 5f84e791335948e180f5575f8174efd1 P 1a490f170c614ee4900ef16c3287c7de reported cstate change: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:30.147341 9828 consensus_queue.cc:237] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } }
I20240502 14:01:30.148612 11098 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.149827 11361 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.151031 11228 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.153043 9695 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } attrs { replace: true } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.154157 9299 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.154881 9150 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f84e791335948e180f5575f8174efd1 with delete_type TABLET_DATA_TOMBSTONED (TS d83af1bc823a4f2ca531f0d6825f8e16 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:37350
I20240502 14:01:30.155117 13747 tablet_replica.cc:330] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16: stopping tablet replica
I20240502 14:01:30.155264 13747 raft_consensus.cc:2229] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:30.155558 13747 raft_consensus.cc:2258] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:30.155637 9826 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.155810 9829 raft_consensus.cc:2932] T 5f84e791335948e180f5575f8174efd1 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.156400 13747 ts_tablet_manager.cc:1875] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:30.156813 9299 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.157378 9830 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.158720 13747 ts_tablet_manager.cc:1888] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 14:01:30.158794 13747 log.cc:1196] T 5f84e791335948e180f5575f8174efd1 P d83af1bc823a4f2ca531f0d6825f8e16: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/wals/5f84e791335948e180f5575f8174efd1
I20240502 14:01:30.159935 8882 catalog_manager.cc:4830] TS d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837): tablet 5f84e791335948e180f5575f8174efd1 (table rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]) successfully deleted
I20240502 14:01:30.160638 9299 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.160732 11228 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.161013 13068 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.161092 9695 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.161550 13731 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.161787 13068 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.161830 9431 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.162719 13731 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.162753 9035 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.165391 13247 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.165766 13731 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.167125 9035 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.167380 9566 consensus_queue.cc:237] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:30.168282 11228 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.170696 9299 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.171166 11099 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.171454 11361 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.171584 13759 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.171891 13760 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.173254 9830 consensus_queue.cc:237] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } }
I20240502 14:01:30.173426 9695 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.173828 13760 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.174367 9431 consensus_queue.cc:237] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:30.174654 9169 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.175050 13760 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.175518 11228 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.175525 9696 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.175984 13068 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.176167 9696 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.176205 8880 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:30.176401 13068 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.176574 13054 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.176815 8894 catalog_manager.cc:5484] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d reported cstate change: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.178181 9036 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.178846 9300 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.179229 9299 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.179325 11099 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.179306 13760 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.179527 13247 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.179594 11097 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.179679 9170 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.179957 9828 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.180227 13247 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.180387 13760 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.180661 12666 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.181003 13068 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.181046 12666 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.181612 9431 raft_consensus.cc:1261] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.181787 9830 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.181874 9302 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.181923 11098 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.182314 13054 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.182669 13054 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.183600 9150 tablet_service.cc:1514] Processing DeleteTablet for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a with delete_type TABLET_DATA_TOMBSTONED (TS d83af1bc823a4f2ca531f0d6825f8e16 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:37350
I20240502 14:01:30.184136 13247 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.184412 13747 tablet_replica.cc:330] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16: stopping tablet replica
I20240502 14:01:30.184577 13747 raft_consensus.cc:2229] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:30.184821 13747 raft_consensus.cc:2258] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:30.185107 13760 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.185592 13747 ts_tablet_manager.cc:1875] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:30.185572 11098 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.185936 9826 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.188094 13747 ts_tablet_manager.cc:1888] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 14:01:30.188161 13747 log.cc:1196] T 97fa16722a1f4adcbf8a5dc3fdfab77a P d83af1bc823a4f2ca531f0d6825f8e16: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/wals/97fa16722a1f4adcbf8a5dc3fdfab77a
I20240502 14:01:30.188406 9697 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.189699 8881 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 21d4acfb937f4dd4b992c338fc6b3237 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:30.189849 9431 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.190325 9695 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.190465 9299 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.191996 11228 raft_consensus.cc:1261] T b1548ca1bcd24d288d30117baea53c12 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.192176 9169 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.192662 13054 consensus_queue.cc:1035] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.194180 8892 catalog_manager.cc:5484] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.194882 13563 consensus_queue.cc:1035] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.194674 13054 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.196578 11099 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.196877 13563 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.198344 9698 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.198891 9830 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.200665 9018 tablet_service.cc:1514] Processing DeleteTablet for tablet 21d4acfb937f4dd4b992c338fc6b3237 with delete_type TABLET_DATA_TOMBSTONED (TS bc9e4f2f668d48ea87d5aad4a2fb569d not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:45946
I20240502 14:01:30.201203 8880 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 6232f47837664d7486612ba1d291f2b8 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:30.201377 8882 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet b1548ca1bcd24d288d30117baea53c12 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:30.201395 9302 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.201583 11230 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.202829 8887 catalog_manager.cc:5484] T b1548ca1bcd24d288d30117baea53c12 P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.202857 9169 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.203167 8887 catalog_manager.cc:5484] T 6232f47837664d7486612ba1d291f2b8 P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:30.204766 9302 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.206979 11228 raft_consensus.cc:2932] T b1548ca1bcd24d288d30117baea53c12 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.206947 8882 catalog_manager.cc:4830] TS d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837): tablet 97fa16722a1f4adcbf8a5dc3fdfab77a (table rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]) successfully deleted
I20240502 14:01:30.209208 9431 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.209719 9018 tablet_service.cc:1514] Processing DeleteTablet for tablet b1548ca1bcd24d288d30117baea53c12 with delete_type TABLET_DATA_TOMBSTONED (TS bc9e4f2f668d48ea87d5aad4a2fb569d not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:45946
I20240502 14:01:30.210054 9150 tablet_service.cc:1514] Processing DeleteTablet for tablet 6232f47837664d7486612ba1d291f2b8 with delete_type TABLET_DATA_TOMBSTONED (TS d83af1bc823a4f2ca531f0d6825f8e16 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:37350
I20240502 14:01:30.211253 9697 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.212687 13747 tablet_replica.cc:330] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16: stopping tablet replica
I20240502 14:01:30.212881 13747 raft_consensus.cc:2229] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Raft consensus shutting down.
W20240502 14:01:30.213637 13853 log.cc:924] Time spent T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d: Append to log took a long time: real 0.090s user 0.001s sys 0.000s
W20240502 14:01:30.214042 13859 log.cc:924] Time spent T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d: Append to log took a long time: real 0.090s user 0.001s sys 0.000s
I20240502 14:01:30.214248 13636 ts_tablet_manager.cc:1875] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20240502 14:01:30.214376 13873 log.cc:924] Time spent T 21d4acfb937f4dd4b992c338fc6b3237 P 304274b88d234198a3892e38c05014a5: Append to log took a long time: real 0.070s user 0.000s sys 0.000s
I20240502 14:01:30.213227 13747 raft_consensus.cc:2258] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:30.215387 13747 ts_tablet_manager.cc:1875] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:30.215747 11100 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.216625 13636 ts_tablet_manager.cc:1888] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 14:01:30.216706 13636 log.cc:1196] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/wals/8c10fb9748c747c6a89066fa113cda3c
I20240502 14:01:30.217386 8882 catalog_manager.cc:4830] TS bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459): tablet 8c10fb9748c747c6a89066fa113cda3c (table rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]) successfully deleted
I20240502 14:01:30.217574 13747 ts_tablet_manager.cc:1888] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 14:01:30.217633 13747 log.cc:1196] T 6232f47837664d7486612ba1d291f2b8 P d83af1bc823a4f2ca531f0d6825f8e16: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-1/wal/wals/6232f47837664d7486612ba1d291f2b8
I20240502 14:01:30.217403 11361 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.217634 13636 tablet_replica.cc:330] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d: stopping tablet replica
I20240502 14:01:30.217754 13636 raft_consensus.cc:2229] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:30.217950 13636 pending_rounds.cc:62] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d: Trying to abort 1 pending ops.
I20240502 14:01:30.218183 13636 pending_rounds.cc:68] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d: Aborting op as it isn't in flight: id { term: 1 index: 9 } timestamp: 7023241175377309696 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "21d4acfb937f4dd4b992c338fc6b3237" old_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } } new_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } } }
I20240502 14:01:30.218427 13636 raft_consensus.cc:2894] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Aborting config change with OpId 1.9: Aborted: Op aborted
I20240502 14:01:30.218500 13636 raft_consensus.cc:2866] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20240502 14:01:30.218559 13636 raft_consensus.cc:2258] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:30.218580 8882 catalog_manager.cc:4830] TS d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837): tablet 6232f47837664d7486612ba1d291f2b8 (table rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]) successfully deleted
I20240502 14:01:30.219105 13636 ts_tablet_manager.cc:1875] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:30.220847 9035 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.221174 13636 ts_tablet_manager.cc:1888] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 14:01:30.221236 13636 log.cc:1196] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/wals/21d4acfb937f4dd4b992c338fc6b3237
I20240502 14:01:30.221592 13636 tablet_replica.cc:330] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d: stopping tablet replica
I20240502 14:01:30.221675 8882 catalog_manager.cc:4830] TS bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459): tablet 21d4acfb937f4dd4b992c338fc6b3237 (table rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]) successfully deleted
I20240502 14:01:30.221774 13636 raft_consensus.cc:2229] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:30.222369 13636 raft_consensus.cc:2258] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:30.223364 11100 raft_consensus.cc:2932] T 6232f47837664d7486612ba1d291f2b8 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:30.225211 13636 ts_tablet_manager.cc:1875] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:30.227203 13636 ts_tablet_manager.cc:1888] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 14:01:30.227335 13636 log.cc:1196] T b1548ca1bcd24d288d30117baea53c12 P bc9e4f2f668d48ea87d5aad4a2fb569d: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/wals/b1548ca1bcd24d288d30117baea53c12
I20240502 14:01:30.228349 8882 catalog_manager.cc:4830] TS bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459): tablet b1548ca1bcd24d288d30117baea53c12 (table rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]) successfully deleted
I20240502 14:01:30.346439 9566 consensus_queue.cc:237] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:30.348237 11228 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.348644 9830 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.348814 9302 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.349231 9169 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.349257 11100 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.349311 13758 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.349627 9697 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:30.349716 13758 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.349963 13762 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.350181 13758 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.350328 13760 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.350911 13760 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:30.352789 13758 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.353101 11100 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.353632 9169 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.355959 8893 catalog_manager.cc:5484] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5 reported cstate change: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.357522 8881 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 7289a5be3b3f437f9a2770baa42b89a8 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:30.357523 9697 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.358242 9302 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.360103 9018 tablet_service.cc:1514] Processing DeleteTablet for tablet 7289a5be3b3f437f9a2770baa42b89a8 with delete_type TABLET_DATA_TOMBSTONED (TS bc9e4f2f668d48ea87d5aad4a2fb569d not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:45946
I20240502 14:01:30.360406 13636 tablet_replica.cc:330] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d: stopping tablet replica
I20240502 14:01:30.360569 13636 raft_consensus.cc:2229] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:30.360873 13636 raft_consensus.cc:2258] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:30.362186 13636 ts_tablet_manager.cc:1875] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:30.363463 9830 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.363729 11228 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:30.364555 13636 ts_tablet_manager.cc:1888] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 14:01:30.364643 13636 log.cc:1196] T 7289a5be3b3f437f9a2770baa42b89a8 P bc9e4f2f668d48ea87d5aad4a2fb569d: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/wals/7289a5be3b3f437f9a2770baa42b89a8
I20240502 14:01:30.365124 8882 catalog_manager.cc:4830] TS bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459): tablet 7289a5be3b3f437f9a2770baa42b89a8 (table rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]) successfully deleted
I20240502 14:01:30.482256 9809 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:01:30.486109 11080 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:30.487452 9412 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:01:30.498323 11342 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:30.500278 9018 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:01:30.511579 9150 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:01:30.516424 9544 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:01:30.531064 9282 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:01:30.533878 11211 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:30.535025 9678 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:30.586095 9169 consensus_queue.cc:237] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:30.587998 11100 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:30.588124 9828 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:30.588124 9697 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
W20240502 14:01:30.588554 9106 consensus_peers.cc:485] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 -> Peer 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74:34421): Couldn't send request to peer 535b19a469c34efaaf49ca1cc2e1e5fc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ada4c2cc1749411d8f41f882b7c64b07. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:30.588586 11228 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:30.588654 13969 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:30.588848 13623 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:30.589206 9566 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:30.589835 13427 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:30.590021 13623 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:30.590236 13969 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:30.590472 9302 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:30.592101 13971 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:30.592546 13971 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:30.592861 9566 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:30.593400 11228 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:30.595208 11100 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:30.597354 9697 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:30.597759 8895 catalog_manager.cc:5484] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca reported cstate change: config changed from index 10 to 11, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:30.602166 9828 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:30.604456 9302 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:31.190613 13987 ts_tablet_manager.cc:902] T ada4c2cc1749411d8f41f882b7c64b07 P 535b19a469c34efaaf49ca1cc2e1e5fc: Initiating tablet copy from peer d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837)
I20240502 14:01:31.191112 13987 tablet_copy_client.cc:308] T ada4c2cc1749411d8f41f882b7c64b07 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.66:38837
I20240502 14:01:31.191601 9190 tablet_copy_service.cc:140] P d83af1bc823a4f2ca531f0d6825f8e16: Received BeginTabletCopySession request for tablet ada4c2cc1749411d8f41f882b7c64b07 from peer 535b19a469c34efaaf49ca1cc2e1e5fc ({username='slave'} at 127.24.66.74:55643)
I20240502 14:01:31.191712 9190 tablet_copy_service.cc:161] P d83af1bc823a4f2ca531f0d6825f8e16: Beginning new tablet copy session on tablet ada4c2cc1749411d8f41f882b7c64b07 from peer 535b19a469c34efaaf49ca1cc2e1e5fc at {username='slave'} at 127.24.66.74:55643: session id = 535b19a469c34efaaf49ca1cc2e1e5fc-ada4c2cc1749411d8f41f882b7c64b07
I20240502 14:01:31.192636 9190 tablet_copy_source_session.cc:205] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:31.193539 13987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ada4c2cc1749411d8f41f882b7c64b07. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:31.195704 13987 tablet_copy_client.cc:783] T ada4c2cc1749411d8f41f882b7c64b07 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:31.195887 13987 tablet_copy_client.cc:647] T ada4c2cc1749411d8f41f882b7c64b07 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:31.197155 13987 tablet_copy_client.cc:523] T ada4c2cc1749411d8f41f882b7c64b07 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:31.198262 13987 tablet_bootstrap.cc:492] T ada4c2cc1749411d8f41f882b7c64b07 P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap starting.
I20240502 14:01:31.219771 13987 tablet_bootstrap.cc:492] T ada4c2cc1749411d8f41f882b7c64b07 P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:31.220064 13987 tablet_bootstrap.cc:492] T ada4c2cc1749411d8f41f882b7c64b07 P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap complete.
I20240502 14:01:31.220222 13987 ts_tablet_manager.cc:1367] T ada4c2cc1749411d8f41f882b7c64b07 P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent bootstrapping tablet: real 0.022s user 0.019s sys 0.000s
I20240502 14:01:31.220664 13987 raft_consensus.cc:348] T ada4c2cc1749411d8f41f882b7c64b07 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:31.220793 13987 raft_consensus.cc:729] T ada4c2cc1749411d8f41f882b7c64b07 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 535b19a469c34efaaf49ca1cc2e1e5fc, State: Initialized, Role: LEARNER
I20240502 14:01:31.220890 13987 consensus_queue.cc:260] T ada4c2cc1749411d8f41f882b7c64b07 P 535b19a469c34efaaf49ca1cc2e1e5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:31.221206 13987 ts_tablet_manager.cc:1398] T ada4c2cc1749411d8f41f882b7c64b07 P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent starting tablet: real 0.001s user 0.000s sys 0.005s
I20240502 14:01:31.221765 9190 tablet_copy_service.cc:342] P d83af1bc823a4f2ca531f0d6825f8e16: Request end of tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-ada4c2cc1749411d8f41f882b7c64b07 received from {username='slave'} at 127.24.66.74:55643
I20240502 14:01:31.221849 9190 tablet_copy_service.cc:433] P d83af1bc823a4f2ca531f0d6825f8e16: ending tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-ada4c2cc1749411d8f41f882b7c64b07 on tablet ada4c2cc1749411d8f41f882b7c64b07 with peer 535b19a469c34efaaf49ca1cc2e1e5fc
I20240502 14:01:31.574270 11361 raft_consensus.cc:1203] T ada4c2cc1749411d8f41f882b7c64b07 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240502 14:01:32.116178 14024 raft_consensus.cc:1050] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16: attempting to promote NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc to VOTER
I20240502 14:01:32.116746 14024 consensus_queue.cc:237] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:32.118909 11230 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:32.119100 11100 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:32.119305 14030 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:32.119540 14029 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:32.119612 9697 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:32.119612 11361 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:32.120033 14028 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:32.120271 9566 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:32.120496 14028 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:32.120605 9302 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:32.121126 9828 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:32.121517 14031 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:32.121984 14029 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:32.122133 14030 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:32.124233 14008 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:32.124603 11360 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:32.126165 11230 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:32.127079 9828 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:32.128011 9566 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:32.128950 8893 catalog_manager.cc:5484] T ada4c2cc1749411d8f41f882b7c64b07 P 535b19a469c34efaaf49ca1cc2e1e5fc reported cstate change: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:32.128952 9697 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:32.130039 9302 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:32.130563 11100 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:32.138383 9169 consensus_queue.cc:237] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:32.139827 9828 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:32.139869 11100 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:32.140100 11360 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:32.140384 14027 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:32.140573 9566 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:32.140753 9697 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:32.140986 14026 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:32.141168 14008 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:32.141342 14029 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:32.141927 14024 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:32.143486 14026 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:32.143895 9566 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:32.145145 11100 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:32.145625 8882 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet ada4c2cc1749411d8f41f882b7c64b07 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:32.146451 9828 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:32.146505 9697 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:32.148721 8887 catalog_manager.cc:5484] T ada4c2cc1749411d8f41f882b7c64b07 P d0f45862d7f14ce1976c1a37842f25ca reported cstate change: config changed from index 12 to 13, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:32.148938 11360 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:32.150182 11230 raft_consensus.cc:1261] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:32.150637 14026 consensus_queue.cc:1035] T ada4c2cc1749411d8f41f882b7c64b07 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:32.151757 11230 raft_consensus.cc:2932] T ada4c2cc1749411d8f41f882b7c64b07 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:32.152618 9282 tablet_service.cc:1514] Processing DeleteTablet for tablet ada4c2cc1749411d8f41f882b7c64b07 with delete_type TABLET_DATA_TOMBSTONED (TS ccfb5bbc1bb54793812d11170ac2d25f not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:54252
I20240502 14:01:32.153033 14047 tablet_replica.cc:330] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f: stopping tablet replica
I20240502 14:01:32.153389 14047 raft_consensus.cc:2229] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:32.153821 14047 raft_consensus.cc:2258] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:32.154536 14047 ts_tablet_manager.cc:1875] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:32.156847 14047 ts_tablet_manager.cc:1888] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240502 14:01:32.156988 14047 log.cc:1196] T ada4c2cc1749411d8f41f882b7c64b07 P ccfb5bbc1bb54793812d11170ac2d25f: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/wals/ada4c2cc1749411d8f41f882b7c64b07
I20240502 14:01:32.157588 8880 catalog_manager.cc:4830] TS ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749): tablet ada4c2cc1749411d8f41f882b7c64b07 (table rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]) successfully deleted
I20240502 14:01:32.343750 9544 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:01:32.344529 9809 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:01:32.347437 9018 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:01:32.356061 9282 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:01:32.358861 11211 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:32.360105 9678 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:32.384898 9150 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:01:32.385040 11342 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:32.385937 11080 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:32.393532 9412 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:01:32.511626 9809 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:01:32.514474 9412 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:01:32.514722 11342 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:32.520812 9544 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:01:32.530664 11080 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:32.542275 9678 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:32.543745 9018 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:01:32.546962 9150 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:01:32.562944 9282 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:01:32.570308 11211 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:32.616443 9169 consensus_queue.cc:237] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:32.618436 9431 consensus_queue.cc:237] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: true } }
W20240502 14:01:32.618770 9105 consensus_peers.cc:485] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 -> Peer 304274b88d234198a3892e38c05014a5 (127.24.66.73:42559): Couldn't send request to peer 304274b88d234198a3892e38c05014a5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3c597826f8094c2e81bebfb87d4c77b5. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:32.618855 9302 consensus_queue.cc:237] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } }
I20240502 14:01:32.618949 11360 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.619319 9697 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.620242 9035 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.620811 11100 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.621387 9302 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.621425 9830 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.621516 11361 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.621763 9828 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.621866 13054 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.622064 14099 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.622246 9566 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.622749 9169 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.623122 9169 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.623327 14024 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.623409 9697 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.623464 14029 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.623772 14099 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20240502 14:01:32.624112 9237 consensus_peers.cc:485] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f -> Peer 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907): Couldn't send request to peer 1a490f170c614ee4900ef16c3287c7de. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:32.624372 9035 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.624375 9036 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.624891 9566 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.625018 13963 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.625322 9828 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.625391 13963 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.625912 14024 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.626228 14024 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.626500 9697 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.626220 14099 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: true } } }
I20240502 14:01:32.626456 13054 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.626789 11360 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.627774 14029 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.627851 9830 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: true } } }
I20240502 14:01:32.628136 14142 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.628504 14142 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.628818 14142 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.629366 9698 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: true } } }
I20240502 14:01:32.630681 8895 catalog_manager.cc:5484] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de reported cstate change: config changed from index 10 to 11, NON_VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) added. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:32.631443 11362 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: true } } }
I20240502 14:01:32.632197 14142 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240502 14:01:32.633105 9367 consensus_peers.cc:485] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de -> Peer ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749): Couldn't send request to peer ccfb5bbc1bb54793812d11170ac2d25f. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:32.633949 14175 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20240502 14:01:32.635001 14174 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.636876 9170 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: true } } }
I20240502 14:01:32.638197 14029 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.639364 11100 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: true } } }
I20240502 14:01:32.639720 9035 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: true } } }
I20240502 14:01:32.641665 14026 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:32.642474 13963 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.643823 9828 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:32.648453 8890 catalog_manager.cc:5484] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:32.649485 9036 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:32.650005 8888 catalog_manager.cc:5484] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:32.651476 11360 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.652599 9038 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.653920 11360 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:32.654796 9299 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:32.655058 9826 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.656793 9695 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.659128 9565 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:32.661628 9697 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } } }
I20240502 14:01:32.664507 9169 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.665151 9565 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.670697 9431 consensus_queue.cc:237] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } }
I20240502 14:01:32.672899 11230 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.673303 14099 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.673738 9830 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
W20240502 14:01:32.673849 9369 consensus_peers.cc:485] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de -> Peer bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459): Couldn't send request to peer bc9e4f2f668d48ea87d5aad4a2fb569d. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:32.673857 9299 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.673868 9169 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.674283 11100 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.674397 9697 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.674927 14174 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.675297 14174 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.676079 14099 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.676399 14099 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.677086 14174 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.682204 14174 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } }
I20240502 14:01:32.684675 9299 consensus_queue.cc:237] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:32.685024 9302 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } }
I20240502 14:01:32.685019 9830 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } }
I20240502 14:01:32.687081 9169 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.687119 11100 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } }
I20240502 14:01:32.687623 9565 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.687885 8891 catalog_manager.cc:5484] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 10 to 11, NON_VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) added. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } }
I20240502 14:01:32.688504 11228 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.688942 9695 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.689060 14178 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.689261 14179 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.690248 9038 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240502 14:01:32.690263 9238 consensus_peers.cc:485] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f -> Peer 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74:34421): Couldn't send request to peer 535b19a469c34efaaf49ca1cc2e1e5fc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0801ed14be134f2a97d33a5c7a8f5cff. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:32.690492 14179 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.690682 14178 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.691001 9168 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } }
I20240502 14:01:32.692202 14179 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.693770 9830 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.694244 14178 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.694420 11230 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } }
I20240502 14:01:32.696971 14142 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.697696 9830 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.698115 9697 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } }
I20240502 14:01:32.699434 9170 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.702039 8892 catalog_manager.cc:5484] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: config changed from index 7 to 8, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.702165 11230 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.704823 9697 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.707142 9566 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.709923 9431 consensus_queue.cc:237] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } }
I20240502 14:01:32.710813 9038 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
W20240502 14:01:32.713321 9368 consensus_peers.cc:485] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de -> Peer d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375): Couldn't send request to peer d0f45862d7f14ce1976c1a37842f25ca. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:32.713383 11100 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:32.713564 11228 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:32.713929 9830 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:32.714519 13054 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:32.714665 14174 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:32.715763 9302 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:32.715858 13054 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:32.716220 9170 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:32.716233 14174 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:32.716840 14174 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:32.716859 9697 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:32.717262 14174 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:32.717254 9566 consensus_queue.cc:237] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } }
W20240502 14:01:32.717895 9369 consensus_peers.cc:485] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de -> Peer bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459): Couldn't send request to peer bc9e4f2f668d48ea87d5aad4a2fb569d. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:32.719729 14239 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } }
W20240502 14:01:32.721475 9501 consensus_peers.cc:485] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca -> Peer bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459): Couldn't send request to peer bc9e4f2f668d48ea87d5aad4a2fb569d. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:32.721848 9302 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } }
I20240502 14:01:32.722131 11360 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.722261 9830 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } }
I20240502 14:01:32.722537 14190 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.725195 9830 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.725585 14000 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.725498 8892 catalog_manager.cc:5484] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: config changed from index 11 to 12, NON_VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) added. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } }
I20240502 14:01:32.725670 11230 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } }
I20240502 14:01:32.726104 11100 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } }
I20240502 14:01:32.726826 9170 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } }
I20240502 14:01:32.729632 11100 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.730020 9302 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.730357 9695 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.730418 14000 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.731303 14190 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.731664 14190 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.733953 14000 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } }
I20240502 14:01:32.734684 9170 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.735031 9697 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, NON_VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } }
I20240502 14:01:32.735288 14190 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.736137 11360 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } }
I20240502 14:01:32.737965 9168 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } }
I20240502 14:01:32.738293 9299 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } }
I20240502 14:01:32.739115 8888 catalog_manager.cc:5484] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc reported cstate change: config changed from index 10 to 11, NON_VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) added. New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } }
I20240502 14:01:32.739401 9695 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } }
I20240502 14:01:32.740710 9830 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } }
I20240502 14:01:32.741799 11100 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } }
I20240502 14:01:32.753365 9302 consensus_queue.cc:237] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:32.754122 9830 consensus_queue.cc:237] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } }
I20240502 14:01:32.755343 11230 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.755904 11100 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.756148 9038 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.756546 9566 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.756880 14180 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.757066 14179 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.757252 14177 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.757401 13963 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.757627 11360 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.757917 9695 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
W20240502 14:01:32.757920 9238 consensus_peers.cc:485] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f -> Peer 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74:34421): Couldn't send request to peer 535b19a469c34efaaf49ca1cc2e1e5fc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bfa2e6af0fd544fa8bfdaefedabb5e68. This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:32.758252 9765 consensus_peers.cc:485] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b -> Peer 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907): Couldn't send request to peer 1a490f170c614ee4900ef16c3287c7de. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:32.758410 9038 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.758598 13999 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.758715 9168 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.759698 14179 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.760174 9565 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.761123 9302 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.761616 14187 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.761974 14187 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.762663 14180 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.763110 9038 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.763360 13999 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.763401 9697 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.763675 13999 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.764804 9695 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.765244 13999 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.765990 11100 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.766757 9830 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.767654 8890 catalog_manager.cc:5484] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d reported cstate change: config changed from index 10 to 11, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.768381 9566 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.770462 11230 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.771226 14268 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.772172 9565 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.772239 9695 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.773737 14177 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.775310 9830 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.779940 8888 catalog_manager.cc:5484] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca reported cstate change: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.784719 9038 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.784060 9168 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.788215 9431 consensus_queue.cc:237] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:32.791052 9695 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:32.791350 11230 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
W20240502 14:01:32.791440 9370 consensus_peers.cc:485] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de -> Peer 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74:34421): Couldn't send request to peer 535b19a469c34efaaf49ca1cc2e1e5fc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8c10fb9748c747c6a89066fa113cda3c. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:32.791656 14297 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:32.792109 9302 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:32.793118 14174 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:32.793208 9168 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:32.793216 11100 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:32.793718 14174 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20240502 14:01:32.794051 9369 consensus_peers.cc:485] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de -> Peer bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459): Couldn't send request to peer bc9e4f2f668d48ea87d5aad4a2fb569d. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20240502 14:01:32.794246 9368 consensus_peers.cc:485] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de -> Peer d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375): Couldn't send request to peer d0f45862d7f14ce1976c1a37842f25ca. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:32.794400 14174 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:32.794826 9830 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:32.795233 13054 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:32.796754 14174 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.797505 13054 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:32.798193 9695 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.799005 9830 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.801489 8888 catalog_manager.cc:5484] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 reported cstate change: config changed from index 12 to 13, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.802233 9302 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.802773 11360 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.804206 11230 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.805497 11100 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.805732 9168 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.808741 9302 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.818995 9830 consensus_queue.cc:237] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:32.821120 9302 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.821308 11230 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.821357 9695 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
W20240502 14:01:32.821550 9766 consensus_peers.cc:485] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b -> Peer 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74:34421): Couldn't send request to peer 535b19a469c34efaaf49ca1cc2e1e5fc. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 97fa16722a1f4adcbf8a5dc3fdfab77a. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:32.821738 14268 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.821976 14270 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.822381 14269 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.822427 11100 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.822624 9038 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.822758 9431 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.823257 14272 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.823580 14272 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.823724 14269 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.826684 14294 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.827747 9302 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.827910 11100 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.830057 9695 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.830145 9038 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.831772 8888 catalog_manager.cc:5484] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: config changed from index 10 to 11, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:32.832826 11230 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.837445 9431 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:32.843750 9168 consensus_queue.cc:237] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } }
I20240502 14:01:32.846009 9431 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.846621 11230 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.847000 9695 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.847607 9830 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.847761 9302 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.848081 14026 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.848872 9038 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.848870 14026 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.849164 14024 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.849493 14024 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240502 14:01:32.849967 9104 consensus_peers.cc:485] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 -> Peer d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375): Couldn't send request to peer d0f45862d7f14ce1976c1a37842f25ca. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:32.850294 14326 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.851657 14326 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.855624 14026 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } }
I20240502 14:01:32.859143 9038 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } }
I20240502 14:01:32.859735 9302 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } }
I20240502 14:01:32.860030 8888 catalog_manager.cc:5484] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: config changed from index 7 to 8, NON_VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) added. New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:32.862963 9830 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } }
I20240502 14:01:32.865792 11230 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } }
I20240502 14:01:32.867574 9695 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } }
I20240502 14:01:32.869490 9431 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } }
I20240502 14:01:32.873827 9038 consensus_queue.cc:237] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } }
I20240502 14:01:32.876520 9302 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.876969 11360 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.877437 14347 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.877660 9826 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
W20240502 14:01:32.877929 8973 consensus_peers.cc:485] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d -> Peer 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907): Couldn't send request to peer 1a490f170c614ee4900ef16c3287c7de. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:32.878036 9168 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.878396 13883 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.878526 9565 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.878731 13883 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.879051 13883 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.879324 13883 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.879983 9695 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:32.880445 13883 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:32.881758 13883 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.882390 9302 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.884256 9695 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.884410 9826 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.884830 8892 catalog_manager.cc:5484] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d reported cstate change: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New cstate: current_term: 1 leader_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:32.886967 11360 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.891641 9565 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.894151 9168 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.902469 9565 consensus_queue.cc:237] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } }
I20240502 14:01:32.904912 9826 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.904984 11230 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.904975 9302 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.905395 11100 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.905388 14190 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.905584 14249 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.905906 14190 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.906060 9168 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
W20240502 14:01:32.908358 9501 consensus_peers.cc:485] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca -> Peer 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907): Couldn't send request to peer 1a490f170c614ee4900ef16c3287c7de. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:32.908685 14190 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.908946 9695 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:32.909049 14190 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.909281 14249 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.909888 11230 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.910444 11100 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.910858 14256 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:32.912077 9168 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.912838 8891 catalog_manager.cc:5484] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca reported cstate change: config changed from index 10 to 11, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:32.912946 9826 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.915408 9302 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:32.916481 9695 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:33.021757 14413 ts_tablet_manager.cc:902] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f: Initiating tablet copy from peer 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907)
I20240502 14:01:33.022508 14413 tablet_copy_client.cc:308] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:42907
I20240502 14:01:33.023106 9454 tablet_copy_service.cc:140] P 1a490f170c614ee4900ef16c3287c7de: Received BeginTabletCopySession request for tablet 9eef89d7e129492bab305a1b5009aace from peer ccfb5bbc1bb54793812d11170ac2d25f ({username='slave'} at 127.24.66.67:56305)
I20240502 14:01:33.023223 9454 tablet_copy_service.cc:161] P 1a490f170c614ee4900ef16c3287c7de: Beginning new tablet copy session on tablet 9eef89d7e129492bab305a1b5009aace from peer ccfb5bbc1bb54793812d11170ac2d25f at {username='slave'} at 127.24.66.67:56305: session id = ccfb5bbc1bb54793812d11170ac2d25f-9eef89d7e129492bab305a1b5009aace
I20240502 14:01:33.024025 9454 tablet_copy_source_session.cc:205] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:33.024979 14413 ts_tablet_manager.cc:1875] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240502 14:01:33.027045 14413 ts_tablet_manager.cc:1888] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.9
I20240502 14:01:33.027196 14413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eef89d7e129492bab305a1b5009aace. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:33.028291 14413 tablet_copy_client.cc:783] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:33.028530 14413 tablet_copy_client.cc:647] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:33.030035 14413 tablet_copy_client.cc:523] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:33.031388 14413 tablet_bootstrap.cc:492] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f: Bootstrap starting.
I20240502 14:01:33.056963 14419 ts_tablet_manager.cc:902] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de: Initiating tablet copy from peer ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:33.057826 14419 tablet_copy_client.cc:308] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.67:34749
I20240502 14:01:33.058394 9322 tablet_copy_service.cc:140] P ccfb5bbc1bb54793812d11170ac2d25f: Received BeginTabletCopySession request for tablet fed16a08a40a474a9159826ca8731acb from peer 1a490f170c614ee4900ef16c3287c7de ({username='slave'} at 127.24.66.68:45369)
I20240502 14:01:33.058521 9322 tablet_copy_service.cc:161] P ccfb5bbc1bb54793812d11170ac2d25f: Beginning new tablet copy session on tablet fed16a08a40a474a9159826ca8731acb from peer 1a490f170c614ee4900ef16c3287c7de at {username='slave'} at 127.24.66.68:45369: session id = 1a490f170c614ee4900ef16c3287c7de-fed16a08a40a474a9159826ca8731acb
I20240502 14:01:33.059551 9322 tablet_copy_source_session.cc:205] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:33.059808 14413 tablet_bootstrap.cc:492] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:33.060221 14413 tablet_bootstrap.cc:492] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f: Bootstrap complete.
I20240502 14:01:33.060514 14413 ts_tablet_manager.cc:1367] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f: Time spent bootstrapping tablet: real 0.029s user 0.024s sys 0.005s
I20240502 14:01:33.060751 14419 ts_tablet_manager.cc:1875] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240502 14:01:33.061169 14413 raft_consensus.cc:348] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: true } }
I20240502 14:01:33.061440 14413 raft_consensus.cc:729] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ccfb5bbc1bb54793812d11170ac2d25f, State: Initialized, Role: LEARNER
I20240502 14:01:33.061609 14413 consensus_queue.cc:260] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: NON_VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: true } }
I20240502 14:01:33.062194 14413 ts_tablet_manager.cc:1398] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:33.063007 9454 tablet_copy_service.cc:342] P 1a490f170c614ee4900ef16c3287c7de: Request end of tablet copy session ccfb5bbc1bb54793812d11170ac2d25f-9eef89d7e129492bab305a1b5009aace received from {username='slave'} at 127.24.66.67:56305
I20240502 14:01:33.063086 9454 tablet_copy_service.cc:433] P 1a490f170c614ee4900ef16c3287c7de: ending tablet copy session ccfb5bbc1bb54793812d11170ac2d25f-9eef89d7e129492bab305a1b5009aace on tablet 9eef89d7e129492bab305a1b5009aace with peer ccfb5bbc1bb54793812d11170ac2d25f
I20240502 14:01:33.063269 14419 ts_tablet_manager.cc:1888] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240502 14:01:33.063460 14419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fed16a08a40a474a9159826ca8731acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:33.065438 14419 tablet_copy_client.cc:783] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:33.065753 14419 tablet_copy_client.cc:647] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:33.068418 14419 tablet_copy_client.cc:523] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:33.070094 14419 tablet_bootstrap.cc:492] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:33.091064 14419 tablet_bootstrap.cc:492] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:33.091403 14419 tablet_bootstrap.cc:492] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de: Bootstrap complete.
I20240502 14:01:33.091593 14419 ts_tablet_manager.cc:1367] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.022s user 0.018s sys 0.001s
I20240502 14:01:33.092015 14419 raft_consensus.cc:348] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } }
I20240502 14:01:33.092141 14419 raft_consensus.cc:729] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: LEARNER
I20240502 14:01:33.092248 14419 consensus_queue.cc:260] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } }
I20240502 14:01:33.092550 14419 ts_tablet_manager.cc:1398] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 14:01:33.093168 9322 tablet_copy_service.cc:342] P ccfb5bbc1bb54793812d11170ac2d25f: Request end of tablet copy session 1a490f170c614ee4900ef16c3287c7de-fed16a08a40a474a9159826ca8731acb received from {username='slave'} at 127.24.66.68:45369
I20240502 14:01:33.093269 9322 tablet_copy_service.cc:433] P ccfb5bbc1bb54793812d11170ac2d25f: ending tablet copy session 1a490f170c614ee4900ef16c3287c7de-fed16a08a40a474a9159826ca8731acb on tablet fed16a08a40a474a9159826ca8731acb with peer 1a490f170c614ee4900ef16c3287c7de
I20240502 14:01:33.173646 14485 ts_tablet_manager.cc:902] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 535b19a469c34efaaf49ca1cc2e1e5fc: Initiating tablet copy from peer ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:33.174222 14485 tablet_copy_client.cc:308] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.67:34749
I20240502 14:01:33.174723 9322 tablet_copy_service.cc:140] P ccfb5bbc1bb54793812d11170ac2d25f: Received BeginTabletCopySession request for tablet bfa2e6af0fd544fa8bfdaefedabb5e68 from peer 535b19a469c34efaaf49ca1cc2e1e5fc ({username='slave'} at 127.24.66.74:58125)
I20240502 14:01:33.174826 9322 tablet_copy_service.cc:161] P ccfb5bbc1bb54793812d11170ac2d25f: Beginning new tablet copy session on tablet bfa2e6af0fd544fa8bfdaefedabb5e68 from peer 535b19a469c34efaaf49ca1cc2e1e5fc at {username='slave'} at 127.24.66.74:58125: session id = 535b19a469c34efaaf49ca1cc2e1e5fc-bfa2e6af0fd544fa8bfdaefedabb5e68
I20240502 14:01:33.175630 9322 tablet_copy_source_session.cc:205] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:33.176806 14485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfa2e6af0fd544fa8bfdaefedabb5e68. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:33.178889 14490 ts_tablet_manager.cc:902] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc: Initiating tablet copy from peer 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907)
I20240502 14:01:33.179261 14485 tablet_copy_client.cc:783] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:33.179394 14490 tablet_copy_client.cc:308] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:42907
I20240502 14:01:33.179457 14485 tablet_copy_client.cc:647] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:33.179822 9454 tablet_copy_service.cc:140] P 1a490f170c614ee4900ef16c3287c7de: Received BeginTabletCopySession request for tablet 8c10fb9748c747c6a89066fa113cda3c from peer 535b19a469c34efaaf49ca1cc2e1e5fc ({username='slave'} at 127.24.66.74:50277)
I20240502 14:01:33.179927 9454 tablet_copy_service.cc:161] P 1a490f170c614ee4900ef16c3287c7de: Beginning new tablet copy session on tablet 8c10fb9748c747c6a89066fa113cda3c from peer 535b19a469c34efaaf49ca1cc2e1e5fc at {username='slave'} at 127.24.66.74:50277: session id = 535b19a469c34efaaf49ca1cc2e1e5fc-8c10fb9748c747c6a89066fa113cda3c
I20240502 14:01:33.180871 14485 tablet_copy_client.cc:523] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:33.180994 9454 tablet_copy_source_session.cc:205] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:33.181962 14490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c10fb9748c747c6a89066fa113cda3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:33.182276 14485 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap starting.
I20240502 14:01:33.182956 14500 ts_tablet_manager.cc:902] T 3c597826f8094c2e81bebfb87d4c77b5 P 304274b88d234198a3892e38c05014a5: Initiating tablet copy from peer d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837)
I20240502 14:01:33.183473 14500 tablet_copy_client.cc:308] T 3c597826f8094c2e81bebfb87d4c77b5 P 304274b88d234198a3892e38c05014a5: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.66:38837
I20240502 14:01:33.184002 14490 tablet_copy_client.cc:783] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:33.183997 9190 tablet_copy_service.cc:140] P d83af1bc823a4f2ca531f0d6825f8e16: Received BeginTabletCopySession request for tablet 3c597826f8094c2e81bebfb87d4c77b5 from peer 304274b88d234198a3892e38c05014a5 ({username='slave'} at 127.24.66.73:52497)
I20240502 14:01:33.184099 9190 tablet_copy_service.cc:161] P d83af1bc823a4f2ca531f0d6825f8e16: Beginning new tablet copy session on tablet 3c597826f8094c2e81bebfb87d4c77b5 from peer 304274b88d234198a3892e38c05014a5 at {username='slave'} at 127.24.66.73:52497: session id = 304274b88d234198a3892e38c05014a5-3c597826f8094c2e81bebfb87d4c77b5
I20240502 14:01:33.184140 14490 tablet_copy_client.cc:647] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:33.185062 9190 tablet_copy_source_session.cc:205] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:33.186205 14500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c597826f8094c2e81bebfb87d4c77b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:33.186416 14490 tablet_copy_client.cc:523] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:33.187621 14490 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap starting.
I20240502 14:01:33.188649 14500 tablet_copy_client.cc:783] T 3c597826f8094c2e81bebfb87d4c77b5 P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:33.188786 14500 tablet_copy_client.cc:647] T 3c597826f8094c2e81bebfb87d4c77b5 P 304274b88d234198a3892e38c05014a5: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:33.189843 14500 tablet_copy_client.cc:523] T 3c597826f8094c2e81bebfb87d4c77b5 P 304274b88d234198a3892e38c05014a5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:33.191006 14500 tablet_bootstrap.cc:492] T 3c597826f8094c2e81bebfb87d4c77b5 P 304274b88d234198a3892e38c05014a5: Bootstrap starting.
I20240502 14:01:33.204691 14512 ts_tablet_manager.cc:902] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d: Initiating tablet copy from peer 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907)
I20240502 14:01:33.205788 14512 tablet_copy_client.cc:308] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:42907
I20240502 14:01:33.206563 9454 tablet_copy_service.cc:140] P 1a490f170c614ee4900ef16c3287c7de: Received BeginTabletCopySession request for tablet 8c10fb9748c747c6a89066fa113cda3c from peer bc9e4f2f668d48ea87d5aad4a2fb569d ({username='slave'} at 127.24.66.65:44731)
I20240502 14:01:33.206674 9454 tablet_copy_service.cc:161] P 1a490f170c614ee4900ef16c3287c7de: Beginning new tablet copy session on tablet 8c10fb9748c747c6a89066fa113cda3c from peer bc9e4f2f668d48ea87d5aad4a2fb569d at {username='slave'} at 127.24.66.65:44731: session id = bc9e4f2f668d48ea87d5aad4a2fb569d-8c10fb9748c747c6a89066fa113cda3c
I20240502 14:01:33.207710 9454 tablet_copy_source_session.cc:205] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:33.213841 14512 ts_tablet_manager.cc:1875] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240502 14:01:33.215730 14485 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:33.216187 14485 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap complete.
I20240502 14:01:33.216430 14485 ts_tablet_manager.cc:1367] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent bootstrapping tablet: real 0.034s user 0.030s sys 0.001s
I20240502 14:01:33.216941 14512 ts_tablet_manager.cc:1888] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.9
I20240502 14:01:33.216977 14485 raft_consensus.cc:348] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:33.217192 14512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c10fb9748c747c6a89066fa113cda3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:33.217401 14485 raft_consensus.cc:729] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 535b19a469c34efaaf49ca1cc2e1e5fc, State: Initialized, Role: LEARNER
I20240502 14:01:33.217590 14485 consensus_queue.cc:260] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 535b19a469c34efaaf49ca1cc2e1e5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:33.217350 14518 ts_tablet_manager.cc:902] T 0801ed14be134f2a97d33a5c7a8f5cff P 535b19a469c34efaaf49ca1cc2e1e5fc: Initiating tablet copy from peer ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:33.218348 14518 tablet_copy_client.cc:308] T 0801ed14be134f2a97d33a5c7a8f5cff P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.67:34749
I20240502 14:01:33.218750 14512 tablet_copy_client.cc:783] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:33.218886 9322 tablet_copy_service.cc:140] P ccfb5bbc1bb54793812d11170ac2d25f: Received BeginTabletCopySession request for tablet 0801ed14be134f2a97d33a5c7a8f5cff from peer 535b19a469c34efaaf49ca1cc2e1e5fc ({username='slave'} at 127.24.66.74:58125)
I20240502 14:01:33.218972 14512 tablet_copy_client.cc:647] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:33.218997 9322 tablet_copy_service.cc:161] P ccfb5bbc1bb54793812d11170ac2d25f: Beginning new tablet copy session on tablet 0801ed14be134f2a97d33a5c7a8f5cff from peer 535b19a469c34efaaf49ca1cc2e1e5fc at {username='slave'} at 127.24.66.74:58125: session id = 535b19a469c34efaaf49ca1cc2e1e5fc-0801ed14be134f2a97d33a5c7a8f5cff
I20240502 14:01:33.219905 9322 tablet_copy_source_session.cc:205] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:33.220405 14512 tablet_copy_client.cc:523] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:33.221290 14485 ts_tablet_manager.cc:1398] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20240502 14:01:33.222023 9322 tablet_copy_service.cc:342] P ccfb5bbc1bb54793812d11170ac2d25f: Request end of tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-bfa2e6af0fd544fa8bfdaefedabb5e68 received from {username='slave'} at 127.24.66.74:58125
I20240502 14:01:33.222096 9322 tablet_copy_service.cc:433] P ccfb5bbc1bb54793812d11170ac2d25f: ending tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-bfa2e6af0fd544fa8bfdaefedabb5e68 on tablet bfa2e6af0fd544fa8bfdaefedabb5e68 with peer 535b19a469c34efaaf49ca1cc2e1e5fc
I20240502 14:01:33.222626 14512 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap starting.
I20240502 14:01:33.224239 14518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0801ed14be134f2a97d33a5c7a8f5cff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:33.225009 14500 tablet_bootstrap.cc:492] T 3c597826f8094c2e81bebfb87d4c77b5 P 304274b88d234198a3892e38c05014a5: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:33.225351 14500 tablet_bootstrap.cc:492] T 3c597826f8094c2e81bebfb87d4c77b5 P 304274b88d234198a3892e38c05014a5: Bootstrap complete.
I20240502 14:01:33.225574 14500 ts_tablet_manager.cc:1367] T 3c597826f8094c2e81bebfb87d4c77b5 P 304274b88d234198a3892e38c05014a5: Time spent bootstrapping tablet: real 0.035s user 0.026s sys 0.005s
I20240502 14:01:33.226080 14500 raft_consensus.cc:348] T 3c597826f8094c2e81bebfb87d4c77b5 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:33.226285 14500 raft_consensus.cc:729] T 3c597826f8094c2e81bebfb87d4c77b5 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 304274b88d234198a3892e38c05014a5, State: Initialized, Role: LEARNER
I20240502 14:01:33.226456 14500 consensus_queue.cc:260] T 3c597826f8094c2e81bebfb87d4c77b5 P 304274b88d234198a3892e38c05014a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: NON_VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: true } }
I20240502 14:01:33.226881 14500 ts_tablet_manager.cc:1398] T 3c597826f8094c2e81bebfb87d4c77b5 P 304274b88d234198a3892e38c05014a5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:33.227036 14518 tablet_copy_client.cc:783] T 0801ed14be134f2a97d33a5c7a8f5cff P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:33.227216 14518 tablet_copy_client.cc:647] T 0801ed14be134f2a97d33a5c7a8f5cff P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:33.227669 14490 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:33.227974 14490 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap complete.
I20240502 14:01:33.228149 14490 ts_tablet_manager.cc:1367] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent bootstrapping tablet: real 0.041s user 0.023s sys 0.007s
I20240502 14:01:33.228986 14518 tablet_copy_client.cc:523] T 0801ed14be134f2a97d33a5c7a8f5cff P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:33.229048 9190 tablet_copy_service.cc:342] P d83af1bc823a4f2ca531f0d6825f8e16: Request end of tablet copy session 304274b88d234198a3892e38c05014a5-3c597826f8094c2e81bebfb87d4c77b5 received from {username='slave'} at 127.24.66.73:52497
I20240502 14:01:33.229135 9190 tablet_copy_service.cc:433] P d83af1bc823a4f2ca531f0d6825f8e16: ending tablet copy session 304274b88d234198a3892e38c05014a5-3c597826f8094c2e81bebfb87d4c77b5 on tablet 3c597826f8094c2e81bebfb87d4c77b5 with peer 304274b88d234198a3892e38c05014a5
I20240502 14:01:33.229319 14490 raft_consensus.cc:348] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:33.229547 14490 raft_consensus.cc:729] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 535b19a469c34efaaf49ca1cc2e1e5fc, State: Initialized, Role: LEARNER
I20240502 14:01:33.229679 14490 consensus_queue.cc:260] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 1.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:33.230283 14490 ts_tablet_manager.cc:1398] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:33.230757 14518 tablet_bootstrap.cc:492] T 0801ed14be134f2a97d33a5c7a8f5cff P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap starting.
I20240502 14:01:33.230840 9454 tablet_copy_service.cc:342] P 1a490f170c614ee4900ef16c3287c7de: Request end of tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-8c10fb9748c747c6a89066fa113cda3c received from {username='slave'} at 127.24.66.74:50277
I20240502 14:01:33.230931 9454 tablet_copy_service.cc:433] P 1a490f170c614ee4900ef16c3287c7de: ending tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-8c10fb9748c747c6a89066fa113cda3c on tablet 8c10fb9748c747c6a89066fa113cda3c with peer 535b19a469c34efaaf49ca1cc2e1e5fc
I20240502 14:01:33.256389 14518 tablet_bootstrap.cc:492] T 0801ed14be134f2a97d33a5c7a8f5cff P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:33.256788 14518 tablet_bootstrap.cc:492] T 0801ed14be134f2a97d33a5c7a8f5cff P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap complete.
I20240502 14:01:33.257011 14518 ts_tablet_manager.cc:1367] T 0801ed14be134f2a97d33a5c7a8f5cff P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent bootstrapping tablet: real 0.026s user 0.019s sys 0.005s
I20240502 14:01:33.257012 14512 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:33.257323 14512 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap complete.
I20240502 14:01:33.257498 14512 ts_tablet_manager.cc:1367] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent bootstrapping tablet: real 0.035s user 0.021s sys 0.004s
I20240502 14:01:33.257519 14518 raft_consensus.cc:348] T 0801ed14be134f2a97d33a5c7a8f5cff P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:33.257663 14518 raft_consensus.cc:729] T 0801ed14be134f2a97d33a5c7a8f5cff P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 535b19a469c34efaaf49ca1cc2e1e5fc, State: Initialized, Role: LEARNER
I20240502 14:01:33.257786 14518 consensus_queue.cc:260] T 0801ed14be134f2a97d33a5c7a8f5cff P 535b19a469c34efaaf49ca1cc2e1e5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:33.257927 14512 raft_consensus.cc:348] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:33.258074 14512 raft_consensus.cc:729] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bc9e4f2f668d48ea87d5aad4a2fb569d, State: Initialized, Role: LEARNER
I20240502 14:01:33.258137 14518 ts_tablet_manager.cc:1398] T 0801ed14be134f2a97d33a5c7a8f5cff P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:33.258175 14512 consensus_queue.cc:260] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 1.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:33.258503 14512 ts_tablet_manager.cc:1398] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:33.258666 9322 tablet_copy_service.cc:342] P ccfb5bbc1bb54793812d11170ac2d25f: Request end of tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-0801ed14be134f2a97d33a5c7a8f5cff received from {username='slave'} at 127.24.66.74:58125
I20240502 14:01:33.258744 9322 tablet_copy_service.cc:433] P ccfb5bbc1bb54793812d11170ac2d25f: ending tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-0801ed14be134f2a97d33a5c7a8f5cff on tablet 0801ed14be134f2a97d33a5c7a8f5cff with peer 535b19a469c34efaaf49ca1cc2e1e5fc
I20240502 14:01:33.259470 9454 tablet_copy_service.cc:342] P 1a490f170c614ee4900ef16c3287c7de: Request end of tablet copy session bc9e4f2f668d48ea87d5aad4a2fb569d-8c10fb9748c747c6a89066fa113cda3c received from {username='slave'} at 127.24.66.65:44731
I20240502 14:01:33.259570 9454 tablet_copy_service.cc:433] P 1a490f170c614ee4900ef16c3287c7de: ending tablet copy session bc9e4f2f668d48ea87d5aad4a2fb569d-8c10fb9748c747c6a89066fa113cda3c on tablet 8c10fb9748c747c6a89066fa113cda3c with peer bc9e4f2f668d48ea87d5aad4a2fb569d
I20240502 14:01:33.265103 14525 ts_tablet_manager.cc:902] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca: Initiating tablet copy from peer d83af1bc823a4f2ca531f0d6825f8e16 (127.24.66.66:38837)
I20240502 14:01:33.266173 14525 tablet_copy_client.cc:308] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.66:38837
I20240502 14:01:33.266770 9190 tablet_copy_service.cc:140] P d83af1bc823a4f2ca531f0d6825f8e16: Received BeginTabletCopySession request for tablet 29eec0daa5d34e40a4ad82607b7ded90 from peer d0f45862d7f14ce1976c1a37842f25ca ({username='slave'} at 127.24.66.69:42341)
I20240502 14:01:33.266875 9190 tablet_copy_service.cc:161] P d83af1bc823a4f2ca531f0d6825f8e16: Beginning new tablet copy session on tablet 29eec0daa5d34e40a4ad82607b7ded90 from peer d0f45862d7f14ce1976c1a37842f25ca at {username='slave'} at 127.24.66.69:42341: session id = d0f45862d7f14ce1976c1a37842f25ca-29eec0daa5d34e40a4ad82607b7ded90
I20240502 14:01:33.267855 9190 tablet_copy_source_session.cc:205] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:33.269759 14525 ts_tablet_manager.cc:1875] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240502 14:01:33.271920 14525 ts_tablet_manager.cc:1888] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240502 14:01:33.272099 14525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29eec0daa5d34e40a4ad82607b7ded90. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:33.273420 14525 tablet_copy_client.cc:783] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:33.273792 14525 tablet_copy_client.cc:647] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:33.275396 14525 tablet_copy_client.cc:523] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:33.277117 14525 tablet_bootstrap.cc:492] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap starting.
I20240502 14:01:33.294014 14541 ts_tablet_manager.cc:902] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca: Initiating tablet copy from peer 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907)
I20240502 14:01:33.294728 14541 tablet_copy_client.cc:308] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:42907
I20240502 14:01:33.296277 9454 tablet_copy_service.cc:140] P 1a490f170c614ee4900ef16c3287c7de: Received BeginTabletCopySession request for tablet 8c10fb9748c747c6a89066fa113cda3c from peer d0f45862d7f14ce1976c1a37842f25ca ({username='slave'} at 127.24.66.69:47239)
I20240502 14:01:33.296423 9454 tablet_copy_service.cc:161] P 1a490f170c614ee4900ef16c3287c7de: Beginning new tablet copy session on tablet 8c10fb9748c747c6a89066fa113cda3c from peer d0f45862d7f14ce1976c1a37842f25ca at {username='slave'} at 127.24.66.69:47239: session id = d0f45862d7f14ce1976c1a37842f25ca-8c10fb9748c747c6a89066fa113cda3c
I20240502 14:01:33.297437 9454 tablet_copy_source_session.cc:205] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:33.298411 14541 ts_tablet_manager.cc:1875] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240502 14:01:33.300503 14419 ts_tablet_manager.cc:902] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de: Initiating tablet copy from peer bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459)
I20240502 14:01:33.301002 14419 tablet_copy_client.cc:308] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.65:45459
I20240502 14:01:33.301582 9058 tablet_copy_service.cc:140] P bc9e4f2f668d48ea87d5aad4a2fb569d: Received BeginTabletCopySession request for tablet f904eef49f9143cd957c8fa5c8ea25ea from peer 1a490f170c614ee4900ef16c3287c7de ({username='slave'} at 127.24.66.68:51737)
I20240502 14:01:33.301697 9058 tablet_copy_service.cc:161] P bc9e4f2f668d48ea87d5aad4a2fb569d: Beginning new tablet copy session on tablet f904eef49f9143cd957c8fa5c8ea25ea from peer 1a490f170c614ee4900ef16c3287c7de at {username='slave'} at 127.24.66.68:51737: session id = 1a490f170c614ee4900ef16c3287c7de-f904eef49f9143cd957c8fa5c8ea25ea
I20240502 14:01:33.302771 9058 tablet_copy_source_session.cc:205] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:33.303623 14419 ts_tablet_manager.cc:1875] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240502 14:01:33.305941 14541 ts_tablet_manager.cc:1888] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240502 14:01:33.306134 14541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c10fb9748c747c6a89066fa113cda3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:33.307492 14419 ts_tablet_manager.cc:1888] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240502 14:01:33.307685 14419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f904eef49f9143cd957c8fa5c8ea25ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:33.308516 14541 tablet_copy_client.cc:783] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:33.308707 14541 tablet_copy_client.cc:647] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:33.309680 14419 tablet_copy_client.cc:783] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:33.309820 14419 tablet_copy_client.cc:647] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:33.310178 14541 tablet_copy_client.cc:523] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:33.312996 14419 tablet_copy_client.cc:523] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:33.313550 14541 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap starting.
I20240502 14:01:33.314404 14419 tablet_bootstrap.cc:492] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:33.311822 14525 tablet_bootstrap.cc:492] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:33.316766 14525 tablet_bootstrap.cc:492] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap complete.
I20240502 14:01:33.316990 14525 ts_tablet_manager.cc:1367] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca: Time spent bootstrapping tablet: real 0.040s user 0.019s sys 0.009s
I20240502 14:01:33.317456 14525 raft_consensus.cc:348] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } }
I20240502 14:01:33.317644 14525 raft_consensus.cc:729] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Initialized, Role: LEARNER
I20240502 14:01:33.317804 14525 consensus_queue.cc:260] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } }
I20240502 14:01:33.318660 14525 ts_tablet_manager.cc:1398] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:33.321008 9190 tablet_copy_service.cc:342] P d83af1bc823a4f2ca531f0d6825f8e16: Request end of tablet copy session d0f45862d7f14ce1976c1a37842f25ca-29eec0daa5d34e40a4ad82607b7ded90 received from {username='slave'} at 127.24.66.69:42341
I20240502 14:01:33.321107 9190 tablet_copy_service.cc:433] P d83af1bc823a4f2ca531f0d6825f8e16: ending tablet copy session d0f45862d7f14ce1976c1a37842f25ca-29eec0daa5d34e40a4ad82607b7ded90 on tablet 29eec0daa5d34e40a4ad82607b7ded90 with peer d0f45862d7f14ce1976c1a37842f25ca
I20240502 14:01:33.334231 14557 ts_tablet_manager.cc:902] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de: Initiating tablet copy from peer d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375)
I20240502 14:01:33.334926 14557 tablet_copy_client.cc:308] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.69:36375
I20240502 14:01:33.335364 9586 tablet_copy_service.cc:140] P d0f45862d7f14ce1976c1a37842f25ca: Received BeginTabletCopySession request for tablet 7289a5be3b3f437f9a2770baa42b89a8 from peer 1a490f170c614ee4900ef16c3287c7de ({username='slave'} at 127.24.66.68:36283)
I20240502 14:01:33.335475 9586 tablet_copy_service.cc:161] P d0f45862d7f14ce1976c1a37842f25ca: Beginning new tablet copy session on tablet 7289a5be3b3f437f9a2770baa42b89a8 from peer 1a490f170c614ee4900ef16c3287c7de at {username='slave'} at 127.24.66.68:36283: session id = 1a490f170c614ee4900ef16c3287c7de-7289a5be3b3f437f9a2770baa42b89a8
I20240502 14:01:33.336066 14512 ts_tablet_manager.cc:902] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d: Initiating tablet copy from peer d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375)
I20240502 14:01:33.336541 9586 tablet_copy_source_session.cc:205] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:33.336544 14512 tablet_copy_client.cc:308] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.69:36375
I20240502 14:01:33.336992 9586 tablet_copy_service.cc:140] P d0f45862d7f14ce1976c1a37842f25ca: Received BeginTabletCopySession request for tablet 21d4acfb937f4dd4b992c338fc6b3237 from peer bc9e4f2f668d48ea87d5aad4a2fb569d ({username='slave'} at 127.24.66.65:59741)
I20240502 14:01:33.337080 9586 tablet_copy_service.cc:161] P d0f45862d7f14ce1976c1a37842f25ca: Beginning new tablet copy session on tablet 21d4acfb937f4dd4b992c338fc6b3237 from peer bc9e4f2f668d48ea87d5aad4a2fb569d at {username='slave'} at 127.24.66.65:59741: session id = bc9e4f2f668d48ea87d5aad4a2fb569d-21d4acfb937f4dd4b992c338fc6b3237
I20240502 14:01:33.337396 14557 ts_tablet_manager.cc:1875] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240502 14:01:33.337843 9586 tablet_copy_source_session.cc:205] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:33.338579 14512 ts_tablet_manager.cc:1875] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240502 14:01:33.339560 14557 ts_tablet_manager.cc:1888] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240502 14:01:33.339790 14557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7289a5be3b3f437f9a2770baa42b89a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:33.340839 14512 ts_tablet_manager.cc:1888] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.9
I20240502 14:01:33.341012 14512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d4acfb937f4dd4b992c338fc6b3237. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:33.341097 14557 tablet_copy_client.cc:783] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:33.341305 14557 tablet_copy_client.cc:647] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:33.342474 14557 tablet_copy_client.cc:523] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:33.342631 14512 tablet_copy_client.cc:783] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:33.342769 14512 tablet_copy_client.cc:647] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:33.344445 14557 tablet_bootstrap.cc:492] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:33.345170 14419 tablet_bootstrap.cc:492] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:33.345328 14512 tablet_copy_client.cc:523] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:33.346576 14419 tablet_bootstrap.cc:492] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de: Bootstrap complete.
I20240502 14:01:33.346791 14419 ts_tablet_manager.cc:1367] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.032s user 0.026s sys 0.000s
I20240502 14:01:33.347271 14419 raft_consensus.cc:348] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } }
I20240502 14:01:33.347424 14419 raft_consensus.cc:729] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: LEARNER
I20240502 14:01:33.347663 14419 consensus_queue.cc:260] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } }
I20240502 14:01:33.348138 14419 ts_tablet_manager.cc:1398] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 14:01:33.348769 14512 tablet_bootstrap.cc:492] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap starting.
I20240502 14:01:33.348922 9058 tablet_copy_service.cc:342] P bc9e4f2f668d48ea87d5aad4a2fb569d: Request end of tablet copy session 1a490f170c614ee4900ef16c3287c7de-f904eef49f9143cd957c8fa5c8ea25ea received from {username='slave'} at 127.24.66.68:51737
I20240502 14:01:33.349009 9058 tablet_copy_service.cc:433] P bc9e4f2f668d48ea87d5aad4a2fb569d: ending tablet copy session 1a490f170c614ee4900ef16c3287c7de-f904eef49f9143cd957c8fa5c8ea25ea on tablet f904eef49f9143cd957c8fa5c8ea25ea with peer 1a490f170c614ee4900ef16c3287c7de
I20240502 14:01:33.350412 14541 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:33.350699 14541 tablet_bootstrap.cc:492] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca: Bootstrap complete.
I20240502 14:01:33.350894 14541 ts_tablet_manager.cc:1367] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca: Time spent bootstrapping tablet: real 0.037s user 0.028s sys 0.000s
I20240502 14:01:33.351308 14541 raft_consensus.cc:348] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:33.351451 14541 raft_consensus.cc:729] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d0f45862d7f14ce1976c1a37842f25ca, State: Initialized, Role: LEARNER
I20240502 14:01:33.351552 14419 ts_tablet_manager.cc:902] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de: Initiating tablet copy from peer 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:33.351552 14541 consensus_queue.cc:260] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 1.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:33.351961 14541 ts_tablet_manager.cc:1398] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 14:01:33.351951 14419 tablet_copy_client.cc:308] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.71:33679
I20240502 14:01:33.352406 9850 tablet_copy_service.cc:140] P 0488b0e4ed994db19d40aed8c15f945b: Received BeginTabletCopySession request for tablet 655c86a7f52a40c68c3e95cb12286c29 from peer 1a490f170c614ee4900ef16c3287c7de ({username='slave'} at 127.24.66.68:47735)
I20240502 14:01:33.352463 9454 tablet_copy_service.cc:342] P 1a490f170c614ee4900ef16c3287c7de: Request end of tablet copy session d0f45862d7f14ce1976c1a37842f25ca-8c10fb9748c747c6a89066fa113cda3c received from {username='slave'} at 127.24.66.69:47239
I20240502 14:01:33.352531 9454 tablet_copy_service.cc:433] P 1a490f170c614ee4900ef16c3287c7de: ending tablet copy session d0f45862d7f14ce1976c1a37842f25ca-8c10fb9748c747c6a89066fa113cda3c on tablet 8c10fb9748c747c6a89066fa113cda3c with peer d0f45862d7f14ce1976c1a37842f25ca
I20240502 14:01:33.352520 9850 tablet_copy_service.cc:161] P 0488b0e4ed994db19d40aed8c15f945b: Beginning new tablet copy session on tablet 655c86a7f52a40c68c3e95cb12286c29 from peer 1a490f170c614ee4900ef16c3287c7de at {username='slave'} at 127.24.66.68:47735: session id = 1a490f170c614ee4900ef16c3287c7de-655c86a7f52a40c68c3e95cb12286c29
I20240502 14:01:33.353585 9850 tablet_copy_source_session.cc:205] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:33.354390 14419 ts_tablet_manager.cc:1875] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240502 14:01:33.356628 14419 ts_tablet_manager.cc:1888] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240502 14:01:33.356797 14419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 655c86a7f52a40c68c3e95cb12286c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:33.358199 14419 tablet_copy_client.cc:783] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:33.358343 14419 tablet_copy_client.cc:647] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:33.359913 14419 tablet_copy_client.cc:523] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:33.361339 14419 tablet_bootstrap.cc:492] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:33.387149 14512 tablet_bootstrap.cc:492] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:33.387490 14512 tablet_bootstrap.cc:492] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d: Bootstrap complete.
I20240502 14:01:33.387717 14512 ts_tablet_manager.cc:1367] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent bootstrapping tablet: real 0.040s user 0.030s sys 0.001s
I20240502 14:01:33.388224 14512 raft_consensus.cc:348] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } }
I20240502 14:01:33.388402 14512 raft_consensus.cc:729] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bc9e4f2f668d48ea87d5aad4a2fb569d, State: Initialized, Role: LEARNER
I20240502 14:01:33.388538 14512 consensus_queue.cc:260] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } }
I20240502 14:01:33.388967 14557 tablet_bootstrap.cc:492] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:33.389256 14557 tablet_bootstrap.cc:492] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap complete.
I20240502 14:01:33.389482 14557 ts_tablet_manager.cc:1367] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.045s user 0.029s sys 0.000s
I20240502 14:01:33.389937 14557 raft_consensus.cc:348] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } }
I20240502 14:01:33.390044 14512 ts_tablet_manager.cc:1398] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:33.390069 14557 raft_consensus.cc:729] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: LEARNER
I20240502 14:01:33.390167 14557 consensus_queue.cc:260] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } }
I20240502 14:01:33.390547 9586 tablet_copy_service.cc:342] P d0f45862d7f14ce1976c1a37842f25ca: Request end of tablet copy session bc9e4f2f668d48ea87d5aad4a2fb569d-21d4acfb937f4dd4b992c338fc6b3237 received from {username='slave'} at 127.24.66.65:59741
I20240502 14:01:33.390630 9586 tablet_copy_service.cc:433] P d0f45862d7f14ce1976c1a37842f25ca: ending tablet copy session bc9e4f2f668d48ea87d5aad4a2fb569d-21d4acfb937f4dd4b992c338fc6b3237 on tablet 21d4acfb937f4dd4b992c338fc6b3237 with peer bc9e4f2f668d48ea87d5aad4a2fb569d
I20240502 14:01:33.391038 14557 ts_tablet_manager.cc:1398] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 14:01:33.392222 9586 tablet_copy_service.cc:342] P d0f45862d7f14ce1976c1a37842f25ca: Request end of tablet copy session 1a490f170c614ee4900ef16c3287c7de-7289a5be3b3f437f9a2770baa42b89a8 received from {username='slave'} at 127.24.66.68:36283
I20240502 14:01:33.392294 9586 tablet_copy_service.cc:433] P d0f45862d7f14ce1976c1a37842f25ca: ending tablet copy session 1a490f170c614ee4900ef16c3287c7de-7289a5be3b3f437f9a2770baa42b89a8 on tablet 7289a5be3b3f437f9a2770baa42b89a8 with peer 1a490f170c614ee4900ef16c3287c7de
I20240502 14:01:33.401803 14419 tablet_bootstrap.cc:492] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:33.402128 14419 tablet_bootstrap.cc:492] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap complete.
I20240502 14:01:33.402348 14419 ts_tablet_manager.cc:1367] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.041s user 0.026s sys 0.000s
I20240502 14:01:33.402829 14419 raft_consensus.cc:348] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } }
I20240502 14:01:33.403028 14419 raft_consensus.cc:729] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: LEARNER
I20240502 14:01:33.403147 14419 consensus_queue.cc:260] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } }
I20240502 14:01:33.403470 14419 ts_tablet_manager.cc:1398] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:33.403965 9850 tablet_copy_service.cc:342] P 0488b0e4ed994db19d40aed8c15f945b: Request end of tablet copy session 1a490f170c614ee4900ef16c3287c7de-655c86a7f52a40c68c3e95cb12286c29 received from {username='slave'} at 127.24.66.68:47735
I20240502 14:01:33.404085 9850 tablet_copy_service.cc:433] P 0488b0e4ed994db19d40aed8c15f945b: ending tablet copy session 1a490f170c614ee4900ef16c3287c7de-655c86a7f52a40c68c3e95cb12286c29 on tablet 655c86a7f52a40c68c3e95cb12286c29 with peer 1a490f170c614ee4900ef16c3287c7de
I20240502 14:01:33.421646 14518 ts_tablet_manager.cc:902] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 535b19a469c34efaaf49ca1cc2e1e5fc: Initiating tablet copy from peer 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:33.422096 14518 tablet_copy_client.cc:308] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.71:33679
I20240502 14:01:33.422602 9850 tablet_copy_service.cc:140] P 0488b0e4ed994db19d40aed8c15f945b: Received BeginTabletCopySession request for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a from peer 535b19a469c34efaaf49ca1cc2e1e5fc ({username='slave'} at 127.24.66.74:53801)
I20240502 14:01:33.422721 9850 tablet_copy_service.cc:161] P 0488b0e4ed994db19d40aed8c15f945b: Beginning new tablet copy session on tablet 97fa16722a1f4adcbf8a5dc3fdfab77a from peer 535b19a469c34efaaf49ca1cc2e1e5fc at {username='slave'} at 127.24.66.74:53801: session id = 535b19a469c34efaaf49ca1cc2e1e5fc-97fa16722a1f4adcbf8a5dc3fdfab77a
I20240502 14:01:33.423710 9850 tablet_copy_source_session.cc:205] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:33.425269 14518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:33.427495 14518 tablet_copy_client.cc:783] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:33.427657 14518 tablet_copy_client.cc:647] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:33.428905 14518 tablet_copy_client.cc:523] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:33.430788 14518 tablet_bootstrap.cc:492] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap starting.
I20240502 14:01:33.456362 14518 tablet_bootstrap.cc:492] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:33.456666 14518 tablet_bootstrap.cc:492] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 535b19a469c34efaaf49ca1cc2e1e5fc: Bootstrap complete.
I20240502 14:01:33.456844 14518 ts_tablet_manager.cc:1367] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent bootstrapping tablet: real 0.026s user 0.024s sys 0.000s
I20240502 14:01:33.457274 14518 raft_consensus.cc:348] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:33.457397 14518 raft_consensus.cc:729] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 535b19a469c34efaaf49ca1cc2e1e5fc, State: Initialized, Role: LEARNER
I20240502 14:01:33.457505 14518 consensus_queue.cc:260] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 535b19a469c34efaaf49ca1cc2e1e5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:33.457784 14518 ts_tablet_manager.cc:1398] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 535b19a469c34efaaf49ca1cc2e1e5fc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:33.458261 9850 tablet_copy_service.cc:342] P 0488b0e4ed994db19d40aed8c15f945b: Request end of tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-97fa16722a1f4adcbf8a5dc3fdfab77a received from {username='slave'} at 127.24.66.74:53801
I20240502 14:01:33.458343 9850 tablet_copy_service.cc:433] P 0488b0e4ed994db19d40aed8c15f945b: ending tablet copy session 535b19a469c34efaaf49ca1cc2e1e5fc-97fa16722a1f4adcbf8a5dc3fdfab77a on tablet 97fa16722a1f4adcbf8a5dc3fdfab77a with peer 535b19a469c34efaaf49ca1cc2e1e5fc
I20240502 14:01:33.573930 9431 raft_consensus.cc:1203] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 14:01:33.633034 9299 raft_consensus.cc:1203] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240502 14:01:33.661568 11360 raft_consensus.cc:1203] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240502 14:01:33.720527 9431 raft_consensus.cc:1203] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240502 14:01:33.721920 9431 raft_consensus.cc:1203] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 14:01:33.735973 11100 raft_consensus.cc:1203] T 3c597826f8094c2e81bebfb87d4c77b5 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 14:01:33.742336 9038 raft_consensus.cc:1203] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 LEARNER]: Deduplicated request from leader. Original: 1.12->[1.13-1.13] Dedup: 1.13->[]
I20240502 14:01:33.744004 11360 raft_consensus.cc:1203] T 0801ed14be134f2a97d33a5c7a8f5cff P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 14:01:33.770503 11360 raft_consensus.cc:1203] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.12->[1.13-1.13] Dedup: 1.13->[]
I20240502 14:01:33.781503 9565 raft_consensus.cc:1203] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 14:01:33.795696 9431 raft_consensus.cc:1203] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 14:01:33.814785 9565 raft_consensus.cc:1203] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEARNER]: Deduplicated request from leader. Original: 1.12->[1.13-1.13] Dedup: 1.13->[]
I20240502 14:01:33.854686 9038 raft_consensus.cc:1203] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240502 14:01:33.898262 11360 raft_consensus.cc:1203] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240502 14:01:34.053881 14177 raft_consensus.cc:1050] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f: attempting to promote NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc to VOTER
I20240502 14:01:34.054440 14177 consensus_queue.cc:237] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:34.056756 11360 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.056754 9826 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.057163 14796 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.057345 9697 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.057682 11100 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.057730 11230 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.058131 14796 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.058378 14177 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.058429 9565 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.058511 14796 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.058724 14797 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.058933 14796 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.059021 9038 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.061461 14177 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.062170 14797 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.062804 11097 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.064831 9826 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.064944 11228 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.065317 9565 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.065718 8891 catalog_manager.cc:5484] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5 reported cstate change: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.067721 11360 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.068480 9038 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.068538 9697 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.071192 14180 raft_consensus.cc:1050] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f: attempting to promote NON_VOTER 1a490f170c614ee4900ef16c3287c7de to VOTER
I20240502 14:01:34.071785 14180 consensus_queue.cc:237] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } }
I20240502 14:01:34.073813 9565 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.073997 9826 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.074142 14796 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.074368 9431 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de [term 1 LEARNER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.074545 9168 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.074687 14179 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.074816 11360 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.075137 9697 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.075225 14796 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.075501 9038 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.075634 14796 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.075803 14179 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.076133 14179 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.077109 14177 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.080951 14177 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.081918 9168 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.083271 9697 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.083308 9826 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.083900 9565 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.084465 9302 consensus_queue.cc:237] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:34.086422 11228 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.086534 11361 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.087067 9036 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.087666 9565 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.087973 14174 raft_consensus.cc:1050] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de: attempting to promote NON_VOTER ccfb5bbc1bb54793812d11170ac2d25f to VOTER
I20240502 14:01:34.088394 14798 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.088577 14174 consensus_queue.cc:237] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: false } }
I20240502 14:01:34.088744 14798 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.089788 8893 catalog_manager.cc:5484] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:34.091565 9302 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEARNER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.091728 11097 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.092061 9697 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.092878 14830 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.093008 9830 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.093266 11360 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.093662 14297 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.094022 14237 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.095355 9431 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.095636 9170 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.095988 14237 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.096879 14812 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.097031 9036 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.097088 14709 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.097795 14812 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.098485 14297 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.099890 9036 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.100942 11361 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.101364 14297 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.102633 9695 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.104048 9299 consensus_queue.cc:237] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } }
I20240502 14:01:34.104743 9826 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.106416 14826 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.107048 9169 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.107234 9565 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.107602 9431 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.107646 14179 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.108068 11362 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.108489 14179 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.108903 14812 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.109364 14180 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.109486 14177 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.111063 9038 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.111472 8880 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet bfa2e6af0fd544fa8bfdaefedabb5e68 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:34.111618 14813 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.111852 9826 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.111950 9564 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.114931 9695 raft_consensus.cc:1261] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.115976 8888 catalog_manager.cc:5484] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca reported cstate change: config changed from index 12 to 13, VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) evicted. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.116717 14179 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.117666 9565 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.118923 14237 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: false } } }
I20240502 14:01:34.119475 8880 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet fed16a08a40a474a9159826ca8731acb with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:34.120435 11097 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: false } } }
I20240502 14:01:34.120553 9038 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.120832 9036 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: false } } }
I20240502 14:01:34.121896 9695 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: false } } }
I20240502 14:01:34.122357 9168 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: false } } }
I20240502 14:01:34.124207 9826 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: false } } }
I20240502 14:01:34.128877 9302 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: false } } }
I20240502 14:01:34.129017 8892 catalog_manager.cc:5484] T fed16a08a40a474a9159826ca8731acb P d0f45862d7f14ce1976c1a37842f25ca reported cstate change: config changed from index 9 to 10, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.129892 9168 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.131112 9036 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.131352 9431 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.131906 8894 catalog_manager.cc:5484] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d reported cstate change: config changed from index 11 to 12, ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: false } } }
I20240502 14:01:34.134090 14179 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.134259 14812 consensus_queue.cc:1035] T fed16a08a40a474a9159826ca8731acb P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.135480 9809 tablet_service.cc:1514] Processing DeleteTablet for tablet fed16a08a40a474a9159826ca8731acb with delete_type TABLET_DATA_TOMBSTONED (TS 0488b0e4ed994db19d40aed8c15f945b not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:33010
I20240502 14:01:34.136024 11360 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.136552 11228 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.137934 14852 tablet_replica.cc:330] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b: stopping tablet replica
I20240502 14:01:34.138384 14852 raft_consensus.cc:2229] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:34.138746 14852 raft_consensus.cc:2258] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:34.138713 11360 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: false } } }
I20240502 14:01:34.138792 9695 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.139649 11361 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.140738 9697 raft_consensus.cc:2932] T fed16a08a40a474a9159826ca8731acb P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.146814 14705 raft_consensus.cc:1050] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d: attempting to promote NON_VOTER 1a490f170c614ee4900ef16c3287c7de to VOTER
I20240502 14:01:34.147378 14705 consensus_queue.cc:237] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } }
I20240502 14:01:34.150264 14852 ts_tablet_manager.cc:1875] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:34.150873 9431 consensus_queue.cc:237] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: false } }
I20240502 14:01:34.151140 9826 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.151898 9565 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.152951 9168 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.152951 11097 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.153168 11361 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.153470 14832 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.153558 9697 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.153869 14832 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.154327 9299 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.154548 14852 ts_tablet_manager.cc:1888] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 14:01:34.154675 14852 log.cc:1196] T fed16a08a40a474a9159826ca8731acb P 0488b0e4ed994db19d40aed8c15f945b: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal/wals/fed16a08a40a474a9159826ca8731acb
I20240502 14:01:34.154647 14830 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.154870 9826 raft_consensus.cc:1261] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.155515 8880 catalog_manager.cc:4830] TS 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679): tablet fed16a08a40a474a9159826ca8731acb (table rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]) successfully deleted
I20240502 14:01:34.155658 14348 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.155803 14830 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.156064 14348 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.157430 14709 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.157744 14237 consensus_queue.cc:1035] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.159790 14831 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: false } } }
I20240502 14:01:34.160164 9830 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: false } } }
I20240502 14:01:34.161437 9168 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.161665 9299 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: false } } }
I20240502 14:01:34.162504 14706 raft_consensus.cc:1050] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16: attempting to promote NON_VOTER d0f45862d7f14ce1976c1a37842f25ca to VOTER
I20240502 14:01:34.163105 14706 consensus_queue.cc:237] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } }
I20240502 14:01:34.163215 11080 tablet_service.cc:1514] Processing DeleteTablet for tablet bfa2e6af0fd544fa8bfdaefedabb5e68 with delete_type TABLET_DATA_TOMBSTONED (TS 304274b88d234198a3892e38c05014a5 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:56756
I20240502 14:01:34.163322 8887 catalog_manager.cc:5484] T 9eef89d7e129492bab305a1b5009aace P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: config changed from index 12 to 13, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: false } } }
I20240502 14:01:34.163872 14867 tablet_replica.cc:330] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5: stopping tablet replica
I20240502 14:01:34.164271 14867 raft_consensus.cc:2229] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:34.164765 14867 raft_consensus.cc:2258] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:34.165581 9299 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.165760 11228 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.166004 14707 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.166213 9829 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.166321 9018 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eef89d7e129492bab305a1b5009aace with delete_type TABLET_DATA_TOMBSTONED (TS bc9e4f2f668d48ea87d5aad4a2fb569d not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:45946
I20240502 14:01:34.166826 9695 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.166867 9564 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEARNER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.167016 14870 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.168012 14326 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.168380 14874 tablet_replica.cc:330] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d: stopping tablet replica
I20240502 14:01:34.168197 14873 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.168576 14874 raft_consensus.cc:2229] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:34.168916 14874 raft_consensus.cc:2258] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:34.169175 9038 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.169363 11360 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.169703 14870 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.169564 11361 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: false } } }
I20240502 14:01:34.170254 9697 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.170583 14874 ts_tablet_manager.cc:1875] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:34.171030 8882 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 9eef89d7e129492bab305a1b5009aace with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:34.171355 9431 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.171650 9430 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de [term 1 LEARNER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.171373 14832 raft_consensus.cc:1050] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de: attempting to promote NON_VOTER bc9e4f2f668d48ea87d5aad4a2fb569d to VOTER
I20240502 14:01:34.172519 14832 consensus_queue.cc:237] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 1.13, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:34.173825 14874 ts_tablet_manager.cc:1888] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240502 14:01:34.174046 14874 log.cc:1196] T 9eef89d7e129492bab305a1b5009aace P bc9e4f2f668d48ea87d5aad4a2fb569d: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-0/wal/wals/9eef89d7e129492bab305a1b5009aace
I20240502 14:01:34.174302 9695 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: false } } }
I20240502 14:01:34.174675 8882 catalog_manager.cc:4830] TS bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65:45459): tablet 9eef89d7e129492bab305a1b5009aace (table rebalance_test_table_1 [id=d4185ca690d0430ba84867fb2b5df872]) successfully deleted
I20240502 14:01:34.175051 9299 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 14:01:34.175352 9829 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 14:01:34.175424 14174 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 14:01:34.175717 14870 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } }
I20240502 14:01:34.176680 9566 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEARNER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 14:01:34.177994 14867 ts_tablet_manager.cc:1875] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:34.178382 11228 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 14:01:34.178419 11230 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } }
I20240502 14:01:34.179159 14297 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 14:01:34.179169 8887 catalog_manager.cc:5484] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: config changed from index 8 to 9, d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:34.179445 14297 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 14:01:34.179646 14348 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.179790 14297 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 14:01:34.179983 14873 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.180037 14348 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.180343 14348 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.181106 9168 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: false } } }
I20240502 14:01:34.181492 9300 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } }
I20240502 14:01:34.181933 9826 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } }
I20240502 14:01:34.182955 9302 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.183579 11361 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 14:01:34.184008 14709 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 14:01:34.184260 14348 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.184479 9698 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } }
I20240502 14:01:34.185638 11100 raft_consensus.cc:2932] T 9eef89d7e129492bab305a1b5009aace P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { promote: false } } }
I20240502 14:01:34.187670 11097 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 14:01:34.188670 14864 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.188920 9695 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 14:01:34.189002 14873 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.189795 14709 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 14:01:34.190197 9036 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 LEARNER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 14:01:34.190611 9038 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } }
I20240502 14:01:34.192382 14867 ts_tablet_manager.cc:1888] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240502 14:01:34.192549 14867 log.cc:1196] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 304274b88d234198a3892e38c05014a5: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/wal/wals/bfa2e6af0fd544fa8bfdaefedabb5e68
I20240502 14:01:34.193218 8882 catalog_manager.cc:4830] TS 304274b88d234198a3892e38c05014a5 (127.24.66.73:42559): tablet bfa2e6af0fd544fa8bfdaefedabb5e68 (table rebalance_test_table_0 [id=5f19c50b051f472e9ea0770bb2938c14]) successfully deleted
I20240502 14:01:34.194659 9434 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } }
I20240502 14:01:34.195845 14237 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 14:01:34.196993 14371 raft_consensus.cc:1050] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca: attempting to promote NON_VOTER 1a490f170c614ee4900ef16c3287c7de to VOTER
I20240502 14:01:34.197595 14371 consensus_queue.cc:237] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } }
I20240502 14:01:34.199304 9430 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEARNER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.199420 9299 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.199862 14256 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.200030 11230 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.200218 14256 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.200585 14297 raft_consensus.cc:1013] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: attempt to promote peer 535b19a469c34efaaf49ca1cc2e1e5fc: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20240502 14:01:34.201148 14297 raft_consensus.cc:1013] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: attempt to promote peer d0f45862d7f14ce1976c1a37842f25ca: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20240502 14:01:34.201511 14249 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.202586 14862 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.205034 9170 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.205439 9302 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.207754 8894 catalog_manager.cc:5484] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d reported cstate change: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:34.209185 14237 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 14:01:34.209807 9168 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 14:01:34.209842 9697 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.210268 11360 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.212091 9697 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.212266 9829 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.212596 14256 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.213227 14249 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.214078 14826 raft_consensus.cc:1050] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f: attempting to promote NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc to VOTER
I20240502 14:01:34.214690 14826 consensus_queue.cc:237] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:34.217653 9696 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.217723 11230 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.218272 14813 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.218654 14813 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.219127 9564 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.219419 9566 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } }
I20240502 14:01:34.219702 11361 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.220638 14177 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.221091 9168 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.221094 14830 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.14: config changed from index 13 to 14, bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:34.221683 14177 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.222862 9168 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.223397 11100 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.223964 14371 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.224022 14177 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.224344 14371 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.224404 11228 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:34.224432 9698 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:34.226656 9302 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:34.226958 9566 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEARNER]: Committing config change with OpId 1.14: config changed from index 13 to 14, bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:34.227852 8887 catalog_manager.cc:5484] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 reported cstate change: config changed from index 13 to 14, bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:34.228929 14237 raft_consensus.cc:1050] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de: attempting to promote NON_VOTER d0f45862d7f14ce1976c1a37842f25ca to VOTER
I20240502 14:01:34.229146 9829 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:34.229482 14237 consensus_queue.cc:237] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 1.14, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:34.231609 11360 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Committing config change with OpId 1.14: config changed from index 13 to 14, bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:34.233345 9565 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEARNER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 14:01:34.233532 11362 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 14:01:34.233970 14922 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 14:01:34.234738 9829 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 14:01:34.234819 11228 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 14:01:34.234984 14829 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 14:01:34.235589 14829 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 14:01:34.236212 9038 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.236289 9035 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 LEARNER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 14:01:34.236703 14830 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 14:01:34.238965 14709 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 14:01:34.239131 9828 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.239390 14709 raft_consensus.cc:1013] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: attempt to promote peer 535b19a469c34efaaf49ca1cc2e1e5fc: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20240502 14:01:34.240123 9168 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 14:01:34.240864 14177 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.240993 9829 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.241420 14813 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.243891 9302 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 14:01:34.244319 9698 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 14:01:34.244470 14174 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 14:01:34.244819 14174 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 14:01:34.246093 14249 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.248803 9566 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.249406 11228 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.251667 9828 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.251956 11098 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 14:01:34.252072 9299 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.253441 14174 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.15: config changed from index 14 to 15, d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:34.255321 8887 catalog_manager.cc:5484] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: config changed from index 11 to 12, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.256095 14297 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 14:01:34.256371 14297 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 14:01:34.257400 9697 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.257448 11360 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Committing config change with OpId 1.15: config changed from index 14 to 15, d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:34.259192 11228 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:34.259917 11098 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:34.260030 9698 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:34.261669 11098 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:34.264597 9168 raft_consensus.cc:858] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }.
Pending config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }:
New config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
W20240502 14:01:34.264695 9168 raft_consensus.cc:1565] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 15' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: NON_VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }.
Pending config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }:
New config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }
I20240502 14:01:34.265081 8893 catalog_manager.cc:5484] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 reported cstate change: config changed from index 14 to 15, d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:34.265169 9168 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:34.265909 9299 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:34.267774 9430 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.268530 14174 raft_consensus.cc:1050] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de: attempting to promote NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc to VOTER
I20240502 14:01:34.269766 9829 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:34.269958 14174 consensus_queue.cc:237] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 1.15, Last appended by leader: 0, Current term: 1, Majority size: 6, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:34.269985 9566 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:34.272487 11360 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240502 14:01:34.273033 14297 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240502 14:01:34.275923 11228 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240502 14:01:34.276306 14831 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240502 14:01:34.276741 14911 raft_consensus.cc:1050] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca: attempting to promote NON_VOTER bc9e4f2f668d48ea87d5aad4a2fb569d to VOTER
I20240502 14:01:34.276945 9829 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240502 14:01:34.277359 14911 consensus_queue.cc:237] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } }
I20240502 14:01:34.277540 14829 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240502 14:01:34.279495 11098 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240502 14:01:34.279598 9299 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240502 14:01:34.280252 11360 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.280706 14371 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.281337 14174 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240502 14:01:34.282094 9169 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.282305 9037 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 LEARNER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.282488 14371 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.282603 9566 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240502 14:01:34.283893 14371 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.284106 9697 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.284457 9299 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.284531 14371 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.284801 14256 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.285351 9829 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.285876 11098 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.286108 14944 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.286557 14830 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240502 14:01:34.286659 14371 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.287032 14174 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240502 14:01:34.287523 11098 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.288266 9698 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240502 14:01:34.288712 14831 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20240502 14:01:34.290447 14269 raft_consensus.cc:1050] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b: attempting to promote NON_VOTER 1a490f170c614ee4900ef16c3287c7de to VOTER
I20240502 14:01:34.290567 14813 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.291033 14269 consensus_queue.cc:237] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } }
I20240502 14:01:34.292057 14830 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.16: config changed from index 15 to 16, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.292301 11100 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.293310 11361 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.293915 11362 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.294718 9430 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEARNER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.295019 9033 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.295100 9301 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.295151 9166 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.295468 9696 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.295468 9300 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.296027 9565 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.296789 14272 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.297506 14269 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.298384 9828 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.299145 8894 catalog_manager.cc:5484] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: config changed from index 8 to 9, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.300299 14268 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.300352 9166 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.300642 14268 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.300771 8892 catalog_manager.cc:5484] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 reported cstate change: config changed from index 15 to 16, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.302407 9167 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240502 14:01:34.303349 9034 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.303918 11228 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.304589 11362 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.305438 9696 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.306505 14174 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 14:01:34.307314 14272 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.307627 14272 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.307787 9167 raft_consensus.cc:858] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }.
Pending config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }:
New config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
W20240502 14:01:34.307857 9167 raft_consensus.cc:1565] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 16' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }.
Pending config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }:
New config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:34.308132 9167 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:34.310151 9566 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.311131 14871 raft_consensus.cc:1050] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16: attempting to promote NON_VOTER 304274b88d234198a3892e38c05014a5 to VOTER
I20240502 14:01:34.311689 14871 consensus_queue.cc:237] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:34.315162 11360 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.315443 9829 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.315696 14706 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.315846 14326 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.316854 9034 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.317267 14371 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } }
I20240502 14:01:34.317670 11100 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P 304274b88d234198a3892e38c05014a5 [term 1 LEARNER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.318270 9302 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.318698 14706 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.319106 9564 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.319509 9826 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } }
I20240502 14:01:34.320031 9302 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } }
I20240502 14:01:34.320679 11362 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } }
I20240502 14:01:34.321302 14326 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.321667 14326 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.323755 9566 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.325582 14706 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.325484 9431 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.326143 9697 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } }
I20240502 14:01:34.327929 9037 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } }
I20240502 14:01:34.328050 9168 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.328953 9828 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.329867 9170 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } }
I20240502 14:01:34.331373 14272 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.331429 9696 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:34.331791 14873 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:34.332553 14706 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:34.332949 9299 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:34.335525 11100 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:34.336620 9037 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:34.338941 11360 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:34.338475 8894 catalog_manager.cc:5484] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:34.339838 8894 catalog_manager.cc:5484] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: config changed from index 11 to 12, bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } }
I20240502 14:01:34.339881 14954 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.340412 9430 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.340759 9829 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:34.342805 9698 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.345124 9565 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.345507 9299 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.350814 8888 catalog_manager.cc:5484] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:34.352522 9696 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:34.353231 9566 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 304274b88d234198a3892e38c05014a5 (127.24.66.73) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:34.353969 11360 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.358207 9033 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.360082 9299 consensus_queue.cc:237] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:34.360954 9033 consensus_queue.cc:237] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } }
I20240502 14:01:34.363353 9170 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.363461 9430 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.363794 9695 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.363837 9696 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.363819 14826 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.364220 14863 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.364429 14798 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.364456 14860 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.364540 11361 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.365401 9168 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.365902 14863 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.365972 11360 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.364977 9299 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.366122 14798 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.366421 14860 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.366551 14348 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.368575 9565 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.368705 9170 consensus_queue.cc:237] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } }
I20240502 14:01:34.368664 14348 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.368953 9299 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.369386 11361 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.370213 9829 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.370664 14871 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.370860 8882 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet f904eef49f9143cd957c8fa5c8ea25ea with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:34.371002 9695 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.371366 14798 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.372426 9299 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.372830 14870 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.373855 8891 catalog_manager.cc:5484] T f904eef49f9143cd957c8fa5c8ea25ea P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 9 to 10, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New cstate: current_term: 1 leader_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.373883 14798 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.374272 11360 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.375213 9564 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.375250 9037 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.375617 14870 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.375936 9696 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.376410 14870 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.376463 11100 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.376897 14870 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.376819 9165 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.379206 8880 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 0801ed14be134f2a97d33a5c7a8f5cff with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:34.379386 9565 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.379544 9430 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.381165 9829 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.382472 14871 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:34.382635 8888 catalog_manager.cc:5484] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 9 to 10, VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) evicted. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:34.382905 9033 raft_consensus.cc:1261] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.383643 14798 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.384037 8882 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 3c597826f8094c2e81bebfb87d4c77b5 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:34.384697 9695 raft_consensus.cc:1261] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.384562 9829 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:34.384757 9564 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:34.385094 14326 consensus_queue.cc:1035] T 3c597826f8094c2e81bebfb87d4c77b5 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.385365 11100 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:34.386503 9809 tablet_service.cc:1514] Processing DeleteTablet for tablet f904eef49f9143cd957c8fa5c8ea25ea with delete_type TABLET_DATA_TOMBSTONED (TS 0488b0e4ed994db19d40aed8c15f945b not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:33010
I20240502 14:01:34.386726 14852 tablet_replica.cc:330] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b: stopping tablet replica
I20240502 14:01:34.386868 14852 raft_consensus.cc:2229] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:34.387116 14852 raft_consensus.cc:2258] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:34.386965 9300 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:34.388032 14852 ts_tablet_manager.cc:1875] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:34.387917 9033 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.388507 14812 consensus_queue.cc:1035] T 0801ed14be134f2a97d33a5c7a8f5cff P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.389532 9695 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:34.390075 9566 raft_consensus.cc:1261] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer bc9e4f2f668d48ea87d5aad4a2fb569d: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.389994 8894 catalog_manager.cc:5484] T 3c597826f8094c2e81bebfb87d4c77b5 P 304274b88d234198a3892e38c05014a5 reported cstate change: config changed from index 9 to 10, VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) evicted. New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:34.390345 14852 ts_tablet_manager.cc:1888] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 14:01:34.390416 14852 log.cc:1196] T f904eef49f9143cd957c8fa5c8ea25ea P 0488b0e4ed994db19d40aed8c15f945b: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal/wals/f904eef49f9143cd957c8fa5c8ea25ea
I20240502 14:01:34.390570 14863 consensus_queue.cc:1035] T f904eef49f9143cd957c8fa5c8ea25ea P bc9e4f2f668d48ea87d5aad4a2fb569d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.391676 9564 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.392141 9037 raft_consensus.cc:2932] T 3c597826f8094c2e81bebfb87d4c77b5 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } }
I20240502 14:01:34.392458 9170 raft_consensus.cc:2932] T f904eef49f9143cd957c8fa5c8ea25ea P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.394052 14272 raft_consensus.cc:1050] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b: attempting to promote NON_VOTER 535b19a469c34efaaf49ca1cc2e1e5fc to VOTER
I20240502 14:01:34.394364 9829 raft_consensus.cc:2932] T 0801ed14be134f2a97d33a5c7a8f5cff P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.394620 14272 consensus_queue.cc:237] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:34.396088 8880 catalog_manager.cc:4830] TS 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679): tablet f904eef49f9143cd957c8fa5c8ea25ea (table rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]) successfully deleted
I20240502 14:01:34.397002 9299 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.397421 14268 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.397861 11100 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.398243 9037 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.398344 14268 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.398799 9696 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.398972 11228 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.399004 14268 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.399672 9430 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.400394 14268 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.400746 14268 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.401119 14272 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.402837 11360 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 LEARNER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:34.403312 14958 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:34.405740 14955 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.407317 9431 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.407953 11100 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.408630 9037 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.408613 11360 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.409554 9430 consensus_queue.cc:237] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 1.16, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:34.411666 9300 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 16. Preceding OpId from leader: term: 1 index: 17. (index mismatch)
I20240502 14:01:34.411643 9696 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.412024 11229 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 16. Preceding OpId from leader: term: 1 index: 17. (index mismatch)
I20240502 14:01:34.412020 11228 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.413125 14174 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240502 14:01:34.413169 8892 catalog_manager.cc:5484] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5 reported cstate change: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.413375 14237 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240502 14:01:34.413878 9566 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 16. Preceding OpId from leader: term: 1 index: 17. (index mismatch)
I20240502 14:01:34.415469 14237 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240502 14:01:34.418896 11100 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 16. Preceding OpId from leader: term: 1 index: 17. (index mismatch)
I20240502 14:01:34.419349 14237 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240502 14:01:34.419747 9566 consensus_queue.cc:237] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } }
I20240502 14:01:34.419893 11360 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 16. Preceding OpId from leader: term: 1 index: 17. (index mismatch)
I20240502 14:01:34.420264 14237 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240502 14:01:34.422278 11097 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.422643 11229 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.423180 9430 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.423123 14935 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.423703 14935 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.425209 9170 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.425109 14174 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.17: config changed from index 16 to 17, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.425668 9696 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.425884 14908 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.426263 14908 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.426824 8882 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 8c10fb9748c747c6a89066fa113cda3c with cas_config_opid_index 16: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:34.427585 9694 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 16. Preceding OpId from leader: term: 1 index: 17. (index mismatch)
W20240502 14:01:34.428828 14915 log.cc:924] Time spent T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7: Append to log took a long time: real 0.139s user 0.000s sys 0.000s
I20240502 14:01:34.429170 14935 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.429459 9165 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 17. (index mismatch)
I20240502 14:01:34.430070 14237 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240502 14:01:34.430271 9035 raft_consensus.cc:1203] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.13->[1.14-1.15] Dedup: 1.14->[1.15-1.15]
I20240502 14:01:34.430370 14237 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20240502 14:01:34.430423 9035 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: NON_VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:34.431444 9694 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.17: config changed from index 16 to 17, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.433595 9038 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 16. (index mismatch)
I20240502 14:01:34.433768 9037 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 1 index: 17. (index mismatch)
I20240502 14:01:34.433544 11360 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.17: config changed from index 16 to 17, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.433821 11228 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.17: config changed from index 16 to 17, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.436138 14829 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 14:01:34.436730 8893 catalog_manager.cc:5484] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 reported cstate change: config changed from index 16 to 17, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.437039 14935 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.437947 9302 raft_consensus.cc:1261] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.438228 9300 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.17: config changed from index 16 to 17, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.439047 11100 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, bc9e4f2f668d48ea87d5aad4a2fb569d (127.24.66.65) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } }
I20240502 14:01:34.439067 9566 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.17: config changed from index 16 to 17, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.440889 9037 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.15: config changed from index 14 to 15, d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } }
I20240502 14:01:34.442085 9165 raft_consensus.cc:858] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }.
Pending config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }:
New config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
W20240502 14:01:34.442178 9165 raft_consensus.cc:1565] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 17' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }.
Pending config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }:
New config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:34.442811 9037 raft_consensus.cc:858] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }.
Pending config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }:
New config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:34.442566 9165 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
W20240502 14:01:34.442888 9037 raft_consensus.cc:1565] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 17' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } }.
Pending config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }:
New config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:34.443349 9037 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.16: config changed from index 15 to 16, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.445746 9697 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.446027 9430 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.447134 14908 consensus_queue.cc:1035] T 7289a5be3b3f437f9a2770baa42b89a8 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.448309 9167 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.448396 11098 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.448988 11228 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.450757 8881 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 7289a5be3b3f437f9a2770baa42b89a8 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:34.451108 9300 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.453799 9165 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.17: config changed from index 16 to 17, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.454505 9566 consensus_queue.cc:237] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } }
I20240502 14:01:34.454447 9037 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.17: config changed from index 16 to 17, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.456368 11342 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c597826f8094c2e81bebfb87d4c77b5 with delete_type TABLET_DATA_TOMBSTONED (TS 535b19a469c34efaaf49ca1cc2e1e5fc not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:51718
I20240502 14:01:34.456259 8891 catalog_manager.cc:5484] T 7289a5be3b3f437f9a2770baa42b89a8 P 304274b88d234198a3892e38c05014a5 reported cstate change: config changed from index 12 to 13, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.456873 9829 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.457401 14945 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.458333 11100 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.17: config changed from index 16 to 17, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } attrs { replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.459749 11360 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.459749 9697 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.460199 14908 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.460813 9809 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c10fb9748c747c6a89066fa113cda3c with delete_type TABLET_DATA_TOMBSTONED (TS 0488b0e4ed994db19d40aed8c15f945b not found in new config with opid_index 17) from {username='slave'} at 127.0.0.1:33010
I20240502 14:01:34.461048 14852 tablet_replica.cc:330] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b: stopping tablet replica
I20240502 14:01:34.461215 14852 raft_consensus.cc:2229] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:34.461658 14852 raft_consensus.cc:2258] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:34.462061 9809 tablet_service.cc:1514] Processing DeleteTablet for tablet 7289a5be3b3f437f9a2770baa42b89a8 with delete_type TABLET_DATA_TOMBSTONED (TS 0488b0e4ed994db19d40aed8c15f945b not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:33010
I20240502 14:01:34.462839 14852 ts_tablet_manager.cc:1875] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:34.463835 14945 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.466935 14852 ts_tablet_manager.cc:1888] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.16
I20240502 14:01:34.467635 9038 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.467659 14852 log.cc:1196] T 8c10fb9748c747c6a89066fa113cda3c P 0488b0e4ed994db19d40aed8c15f945b: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal/wals/8c10fb9748c747c6a89066fa113cda3c
I20240502 14:01:34.468086 11100 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.467471 14944 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } }
I20240502 14:01:34.468533 14256 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.468765 14852 tablet_replica.cc:330] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b: stopping tablet replica
I20240502 14:01:34.468907 14852 raft_consensus.cc:2229] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:34.469240 14852 raft_consensus.cc:2258] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:34.469079 11360 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } }
I20240502 14:01:34.469982 14852 ts_tablet_manager.cc:1875] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:34.470491 9829 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } }
I20240502 14:01:34.470692 9695 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } }
I20240502 14:01:34.471808 8890 catalog_manager.cc:5484] T 21d4acfb937f4dd4b992c338fc6b3237 P 535b19a469c34efaaf49ca1cc2e1e5fc reported cstate change: config changed from index 12 to 13, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New cstate: current_term: 1 leader_uuid: "d0f45862d7f14ce1976c1a37842f25ca" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } }
I20240502 14:01:34.472793 14935 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.473273 9169 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { replace: true } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.473368 9697 raft_consensus.cc:2932] T 7289a5be3b3f437f9a2770baa42b89a8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.477038 14852 ts_tablet_manager.cc:1888] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240502 14:01:34.477114 14852 log.cc:1196] T 7289a5be3b3f437f9a2770baa42b89a8 P 0488b0e4ed994db19d40aed8c15f945b: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal/wals/7289a5be3b3f437f9a2770baa42b89a8
I20240502 14:01:34.477514 8881 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 21d4acfb937f4dd4b992c338fc6b3237 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:34.478060 9038 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } }
I20240502 14:01:34.478235 9430 consensus_queue.cc:237] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 1.17, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:34.479399 11100 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } }
I20240502 14:01:34.481534 11360 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 17. Preceding OpId from leader: term: 1 index: 18. (index mismatch)
I20240502 14:01:34.481809 9169 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 17. Preceding OpId from leader: term: 1 index: 18. (index mismatch)
I20240502 14:01:34.481976 14923 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20240502 14:01:34.482277 11098 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 17. Preceding OpId from leader: term: 1 index: 18. (index mismatch)
I20240502 14:01:34.482759 14709 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20240502 14:01:34.482939 8880 catalog_manager.cc:4830] TS 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679): tablet 8c10fb9748c747c6a89066fa113cda3c (table rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]) successfully deleted
I20240502 14:01:34.483053 14829 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20240502 14:01:34.484987 9566 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 17. Preceding OpId from leader: term: 1 index: 18. (index mismatch)
I20240502 14:01:34.485572 11228 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 17. Preceding OpId from leader: term: 1 index: 18. (index mismatch)
I20240502 14:01:34.485955 9695 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 17. Preceding OpId from leader: term: 1 index: 18. (index mismatch)
I20240502 14:01:34.485997 14829 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20240502 14:01:34.486315 14709 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20240502 14:01:34.486425 14829 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20240502 14:01:34.487800 9035 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 17. Preceding OpId from leader: term: 1 index: 18. (index mismatch)
I20240502 14:01:34.488481 14297 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.18: config changed from index 17 to 18, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.489519 9165 raft_consensus.cc:1261] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer d0f45862d7f14ce1976c1a37842f25ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.489830 14829 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20240502 14:01:34.490398 8882 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 8c10fb9748c747c6a89066fa113cda3c with cas_config_opid_index 17: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:34.490398 8880 catalog_manager.cc:4830] TS 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679): tablet 7289a5be3b3f437f9a2770baa42b89a8 (table rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]) successfully deleted
I20240502 14:01:34.491051 14935 consensus_queue.cc:1035] T 21d4acfb937f4dd4b992c338fc6b3237 P d0f45862d7f14ce1976c1a37842f25ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.491313 8887 catalog_manager.cc:5484] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de reported cstate change: config changed from index 17 to 18, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:34.491674 11360 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.18: config changed from index 17 to 18, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.492316 9169 raft_consensus.cc:2932] T 21d4acfb937f4dd4b992c338fc6b3237 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } }
I20240502 14:01:34.492446 11228 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.18: config changed from index 17 to 18, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.492575 11098 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.18: config changed from index 17 to 18, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.493656 9037 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.18: config changed from index 17 to 18, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.494714 14995 tablet_replica.cc:330] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc: stopping tablet replica
I20240502 14:01:34.495491 9165 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.18: config changed from index 17 to 18, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.495939 14995 raft_consensus.cc:2229] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:34.496881 14995 raft_consensus.cc:2258] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:34.499440 9566 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.18: config changed from index 17 to 18, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.500309 14995 ts_tablet_manager.cc:1875] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:34.502460 9282 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c10fb9748c747c6a89066fa113cda3c with delete_type TABLET_DATA_TOMBSTONED (TS ccfb5bbc1bb54793812d11170ac2d25f not found in new config with opid_index 18) from {username='slave'} at 127.0.0.1:54252
I20240502 14:01:34.502540 9281 tablet_service.cc:1514] Processing DeleteTablet for tablet 21d4acfb937f4dd4b992c338fc6b3237 with delete_type TABLET_DATA_TOMBSTONED (TS ccfb5bbc1bb54793812d11170ac2d25f not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:54252
I20240502 14:01:34.503540 15000 tablet_replica.cc:330] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f: stopping tablet replica
I20240502 14:01:34.503752 15000 raft_consensus.cc:2229] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:34.504281 15000 raft_consensus.cc:2258] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:34.505338 9829 consensus_queue.cc:237] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } }
I20240502 14:01:34.505836 15000 ts_tablet_manager.cc:1875] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:34.507879 9300 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.508543 14272 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.509047 11360 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.509662 9697 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.509886 9037 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.510139 14269 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.510293 14956 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.510421 14955 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.513072 9430 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.513373 9165 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.513411 14272 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.513903 9695 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.18: config changed from index 17 to 18, VOTER ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.514372 14955 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.515432 9165 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.516377 9300 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.518369 14268 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.517623 8891 catalog_manager.cc:5484] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b reported cstate change: config changed from index 9 to 10, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:34.519438 9430 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.519963 11360 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.521103 9037 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.526242 9544 tablet_service.cc:1514] Processing DeleteTablet for tablet 655c86a7f52a40c68c3e95cb12286c29 with delete_type TABLET_DATA_TOMBSTONED (TS d0f45862d7f14ce1976c1a37842f25ca not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:46242
I20240502 14:01:34.526850 8880 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 655c86a7f52a40c68c3e95cb12286c29 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:34.529001 15006 tablet_replica.cc:330] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca: stopping tablet replica
I20240502 14:01:34.529140 15006 raft_consensus.cc:2229] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:34.528972 15000 ts_tablet_manager.cc:1888] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.17
I20240502 14:01:34.529402 15006 raft_consensus.cc:2258] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:34.529510 15000 log.cc:1196] T 8c10fb9748c747c6a89066fa113cda3c P ccfb5bbc1bb54793812d11170ac2d25f: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/wals/8c10fb9748c747c6a89066fa113cda3c
I20240502 14:01:34.530134 15000 tablet_replica.cc:330] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f: stopping tablet replica
I20240502 14:01:34.530171 8880 catalog_manager.cc:4830] TS ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749): tablet 8c10fb9748c747c6a89066fa113cda3c (table rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]) successfully deleted
I20240502 14:01:34.530380 15000 raft_consensus.cc:2229] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:34.530905 15006 ts_tablet_manager.cc:1875] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:34.530949 15000 raft_consensus.cc:2258] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:34.531394 14995 ts_tablet_manager.cc:1888] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 14:01:34.531498 14995 log.cc:1196] T 3c597826f8094c2e81bebfb87d4c77b5 P 535b19a469c34efaaf49ca1cc2e1e5fc: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/wal/wals/3c597826f8094c2e81bebfb87d4c77b5
I20240502 14:01:34.531803 15000 ts_tablet_manager.cc:1875] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:34.532039 8883 catalog_manager.cc:4830] TS 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74:34421): tablet 3c597826f8094c2e81bebfb87d4c77b5 (table rebalance_test_table_2 [id=404e10152ca84ba19a1b445d8a4f708c]) successfully deleted
I20240502 14:01:34.533365 15006 ts_tablet_manager.cc:1888] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 14:01:34.533501 15006 log.cc:1196] T 655c86a7f52a40c68c3e95cb12286c29 P d0f45862d7f14ce1976c1a37842f25ca: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-4/wal/wals/655c86a7f52a40c68c3e95cb12286c29
I20240502 14:01:34.534231 8881 catalog_manager.cc:4830] TS d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69:36375): tablet 655c86a7f52a40c68c3e95cb12286c29 (table rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]) successfully deleted
I20240502 14:01:34.534895 15000 ts_tablet_manager.cc:1888] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240502 14:01:34.535066 15000 log.cc:1196] T 21d4acfb937f4dd4b992c338fc6b3237 P ccfb5bbc1bb54793812d11170ac2d25f: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-2/wal/wals/21d4acfb937f4dd4b992c338fc6b3237
I20240502 14:01:34.535547 11211 tablet_service.cc:1514] Processing DeleteTablet for tablet 0801ed14be134f2a97d33a5c7a8f5cff with delete_type TABLET_DATA_TOMBSTONED (TS e01cf117ef5d446590c06dd1a6ab7ef7 not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:46242
I20240502 14:01:34.535519 9695 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER d0f45862d7f14ce1976c1a37842f25ca (127.24.66.69) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } }
I20240502 14:01:34.537173 8880 catalog_manager.cc:4830] TS ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749): tablet 21d4acfb937f4dd4b992c338fc6b3237 (table rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]) successfully deleted
I20240502 14:01:34.537819 15014 tablet_replica.cc:330] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7: stopping tablet replica
I20240502 14:01:34.538290 15014 raft_consensus.cc:2229] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:34.538756 15014 pending_rounds.cc:62] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7: Trying to abort 1 pending ops.
I20240502 14:01:34.538866 15014 pending_rounds.cc:68] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7: Aborting op as it isn't in flight: id { term: 1 index: 9 } timestamp: 7023241192301699072 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "0801ed14be134f2a97d33a5c7a8f5cff" old_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: NON_VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: true } } } new_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false replace: true } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } } }
I20240502 14:01:34.539232 15014 raft_consensus.cc:2894] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Aborting config change with OpId 1.9: Aborted: Op aborted
I20240502 14:01:34.539378 15014 raft_consensus.cc:2866] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20240502 14:01:34.539484 15014 raft_consensus.cc:2258] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:34.540752 15014 ts_tablet_manager.cc:1875] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:34.543255 15014 ts_tablet_manager.cc:1888] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 14:01:34.543398 15014 log.cc:1196] T 0801ed14be134f2a97d33a5c7a8f5cff P e01cf117ef5d446590c06dd1a6ab7ef7: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal/wals/0801ed14be134f2a97d33a5c7a8f5cff
I20240502 14:01:34.544162 8883 catalog_manager.cc:4830] TS e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72:35005): tablet 0801ed14be134f2a97d33a5c7a8f5cff (table rebalance_test_table_6 [id=a473c7db0a6f423c8ca6c07ba216cbb3]) successfully deleted
I20240502 14:01:34.546084 9299 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.552690 9829 consensus_queue.cc:237] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:34.555064 11360 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.555585 11228 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.555994 9430 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.555944 14956 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.556419 14956 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.556593 9037 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.556630 9299 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.556985 14953 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.557844 14953 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.558362 14269 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.560341 9695 raft_consensus.cc:1261] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:34.560485 14954 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.561229 9037 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.560794 14269 consensus_queue.cc:1035] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:34.565809 11228 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.565943 9299 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.566500 8880 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:34.567471 9430 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.568279 11360 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.569456 8892 catalog_manager.cc:5484] T 97fa16722a1f4adcbf8a5dc3fdfab77a P e01cf117ef5d446590c06dd1a6ab7ef7 reported cstate change: config changed from index 12 to 13, VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) evicted. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.574538 9829 consensus_queue.cc:237] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:34.576726 11360 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
W20240502 14:01:34.576817 9766 consensus_peers.cc:485] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b -> Peer e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72:35005): Couldn't send request to peer e01cf117ef5d446590c06dd1a6ab7ef7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 655c86a7f52a40c68c3e95cb12286c29. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:34.576874 9697 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:34.577353 9430 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:34.577697 9037 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:34.577742 14269 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:34.578042 14268 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:34.578125 9165 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:34.578678 9299 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20240502 14:01:34.579063 14956 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:34.579350 14956 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:34.579424 9300 consensus_queue.cc:237] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } }
I20240502 14:01:34.579622 14956 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:34.581115 11080 tablet_service.cc:1514] Processing DeleteTablet for tablet 97fa16722a1f4adcbf8a5dc3fdfab77a with delete_type TABLET_DATA_TOMBSTONED (TS 304274b88d234198a3892e38c05014a5 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:56756
I20240502 14:01:34.581390 14867 tablet_replica.cc:330] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5: stopping tablet replica
I20240502 14:01:34.581529 14867 raft_consensus.cc:2229] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:34.581775 14867 raft_consensus.cc:2258] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:34.582885 9566 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:34.583284 14812 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:34.583487 9165 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:34.583302 9695 raft_consensus.cc:2932] T 97fa16722a1f4adcbf8a5dc3fdfab77a P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 304274b88d234198a3892e38c05014a5 (127.24.66.73) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:34.584111 9697 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:34.584350 11228 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:34.584465 14867 ts_tablet_manager.cc:1875] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:34.584442 14812 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:34.584424 14956 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:34.585036 14798 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:34.585330 14798 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:34.586814 14867 ts_tablet_manager.cc:1888] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240502 14:01:34.586889 14867 log.cc:1196] T 97fa16722a1f4adcbf8a5dc3fdfab77a P 304274b88d234198a3892e38c05014a5: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-8/wal/wals/97fa16722a1f4adcbf8a5dc3fdfab77a
I20240502 14:01:34.587466 8882 catalog_manager.cc:4830] TS 304274b88d234198a3892e38c05014a5 (127.24.66.73:42559): tablet 97fa16722a1f4adcbf8a5dc3fdfab77a (table rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]) successfully deleted
W20240502 14:01:34.587903 9237 consensus_peers.cc:485] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f -> Peer 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907): Couldn't send request to peer 1a490f170c614ee4900ef16c3287c7de. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:34.588069 9299 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:34.588634 9037 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:34.589231 14812 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:34.589354 14955 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20240502 14:01:34.590240 9698 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:34.590885 9169 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:34.591101 9035 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:34.593864 8891 catalog_manager.cc:5484] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d reported cstate change: config changed from index 10 to 11, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:34.595566 9829 raft_consensus.cc:1261] T 50451fdbd43b491fa701daac5de5e1c8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:34.599469 11360 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:34.601090 9430 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 10 to 11, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:34.603729 14812 consensus_queue.cc:1035] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:34.606427 14796 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:34.607653 9430 consensus_queue.cc:237] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:34.609091 9566 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:34.609578 9037 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:34.610560 9829 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:34.612213 8888 catalog_manager.cc:5484] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:34.613927 9169 consensus_queue.cc:237] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } }
I20240502 14:01:34.615468 9299 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:34.615944 14922 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:34.616263 11229 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.617089 11360 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:34.617067 14872 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.617976 9694 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:34.618468 14922 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:34.618641 14923 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:34.618773 9037 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:34.619200 14923 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20240502 14:01:34.619875 9370 consensus_peers.cc:485] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de -> Peer e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72:35005): Couldn't send request to peer e01cf117ef5d446590c06dd1a6ab7ef7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6aed24ef77e54c8b8ff09e6709a886e4. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:34.621430 9430 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.621779 9167 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:34.622018 9035 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.622865 14872 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.623142 9566 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.623149 14872 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.623698 9698 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.623739 14326 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.624135 14923 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:34.624276 9300 raft_consensus.cc:1261] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer d83af1bc823a4f2ca531f0d6825f8e16: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20240502 14:01:34.624250 14326 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.624915 14872 consensus_queue.cc:1035] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 14:01:34.625392 14923 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:34.627733 9299 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:34.628134 11360 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:34.628821 14326 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } }
I20240502 14:01:34.631547 8887 catalog_manager.cc:5484] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:34.632172 9035 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } }
I20240502 14:01:34.632613 9566 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } }
I20240502 14:01:34.632875 11230 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } }
I20240502 14:01:34.635668 8890 catalog_manager.cc:5484] T 29eec0daa5d34e40a4ad82607b7ded90 P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: config changed from index 9 to 10, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New cstate: current_term: 1 leader_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:34.637151 8882 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 29eec0daa5d34e40a4ad82607b7ded90 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:34.637419 9829 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20240502 14:01:34.638947 9698 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } }
I20240502 14:01:34.639305 9430 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } }
I20240502 14:01:34.639467 14829 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 14:01:34.641628 9694 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:34.643581 9809 tablet_service.cc:1514] Processing DeleteTablet for tablet 29eec0daa5d34e40a4ad82607b7ded90 with delete_type TABLET_DATA_TOMBSTONED (TS 0488b0e4ed994db19d40aed8c15f945b not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:33010
I20240502 14:01:34.643802 14852 tablet_replica.cc:330] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b: stopping tablet replica
I20240502 14:01:34.643947 14852 raft_consensus.cc:2229] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:34.643903 9829 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:34.644178 14852 raft_consensus.cc:2258] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:34.645038 14852 ts_tablet_manager.cc:1875] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:34.645771 9300 raft_consensus.cc:2932] T 29eec0daa5d34e40a4ad82607b7ded90 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 9 to 10, VOTER 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } }
I20240502 14:01:34.647485 14852 ts_tablet_manager.cc:1888] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.9
I20240502 14:01:34.648445 14852 log.cc:1196] T 29eec0daa5d34e40a4ad82607b7ded90 P 0488b0e4ed994db19d40aed8c15f945b: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-6/wal/wals/29eec0daa5d34e40a4ad82607b7ded90
I20240502 14:01:34.649817 8880 catalog_manager.cc:4830] TS 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679): tablet 29eec0daa5d34e40a4ad82607b7ded90 (table rebalance_test_table_4 [id=44e1a9d10fe5492aa8fb6f4e25c523e4]) successfully deleted
I20240502 14:01:34.655558 9037 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:34.665648 11230 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:34.668423 9697 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:34.669067 9165 raft_consensus.cc:2932] T 50451fdbd43b491fa701daac5de5e1c8 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:34.672904 9165 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } } }
I20240502 14:01:34.679416 9300 consensus_queue.cc:237] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 1.13, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } }
I20240502 14:01:34.681685 9037 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
W20240502 14:01:34.681718 9237 consensus_peers.cc:485] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f -> Peer 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907): Couldn't send request to peer 1a490f170c614ee4900ef16c3287c7de. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240502 14:01:34.682188 14797 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 14:01:34.682278 9829 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 14:01:34.682679 14797 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 14:01:34.683787 11360 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 14:01:34.684510 14797 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 14:01:34.685251 9566 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 14:01:34.685706 14797 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 14:01:34.687801 14179 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 LEADER]: Committing config change with OpId 1.14: config changed from index 13 to 14, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:34.689083 11228 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 14:01:34.689177 9697 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 1 index: 14. (index mismatch)
I20240502 14:01:34.690372 14180 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 14:01:34.690671 14826 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 14:01:34.690759 9037 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:34.690949 8887 catalog_manager.cc:5484] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f reported cstate change: config changed from index 13 to 14, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New cstate: current_term: 1 leader_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20240502 14:01:34.691824 11360 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:34.692186 11228 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:34.694559 9829 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:34.698717 9566 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:34.698830 9697 raft_consensus.cc:2932] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.14: config changed from index 13 to 14, NON_VOTER 1a490f170c614ee4900ef16c3287c7de (127.24.66.68) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } } }
I20240502 14:01:34.758236 9412 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:01:34.760135 9809 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 14:01:34.767779 9018 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 14:01:34.770720 9282 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 14:01:34.779824 11080 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:34.795784 9678 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:34.803514 11342 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:34.815120 9150 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:01:34.821336 11211 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 14:01:34.844761 9544 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 14:01:35.016553 15177 ts_tablet_manager.cc:902] T 6aed24ef77e54c8b8ff09e6709a886e4 P e01cf117ef5d446590c06dd1a6ab7ef7: Initiating tablet copy from peer 1a490f170c614ee4900ef16c3287c7de (127.24.66.68:42907)
I20240502 14:01:35.017311 15177 tablet_copy_client.cc:308] T 6aed24ef77e54c8b8ff09e6709a886e4 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.68:42907
I20240502 14:01:35.017812 9454 tablet_copy_service.cc:140] P 1a490f170c614ee4900ef16c3287c7de: Received BeginTabletCopySession request for tablet 6aed24ef77e54c8b8ff09e6709a886e4 from peer e01cf117ef5d446590c06dd1a6ab7ef7 ({username='slave'} at 127.24.66.72:40361)
I20240502 14:01:35.017920 9454 tablet_copy_service.cc:161] P 1a490f170c614ee4900ef16c3287c7de: Beginning new tablet copy session on tablet 6aed24ef77e54c8b8ff09e6709a886e4 from peer e01cf117ef5d446590c06dd1a6ab7ef7 at {username='slave'} at 127.24.66.72:40361: session id = e01cf117ef5d446590c06dd1a6ab7ef7-6aed24ef77e54c8b8ff09e6709a886e4
I20240502 14:01:35.018680 9454 tablet_copy_source_session.cc:205] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:35.019642 15177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aed24ef77e54c8b8ff09e6709a886e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:35.022189 15177 tablet_copy_client.cc:783] T 6aed24ef77e54c8b8ff09e6709a886e4 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:35.022368 15177 tablet_copy_client.cc:647] T 6aed24ef77e54c8b8ff09e6709a886e4 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:35.023948 15177 tablet_copy_client.cc:523] T 6aed24ef77e54c8b8ff09e6709a886e4 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:35.025611 15177 tablet_bootstrap.cc:492] T 6aed24ef77e54c8b8ff09e6709a886e4 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap starting.
I20240502 14:01:35.049714 15177 tablet_bootstrap.cc:492] T 6aed24ef77e54c8b8ff09e6709a886e4 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:35.050060 15177 tablet_bootstrap.cc:492] T 6aed24ef77e54c8b8ff09e6709a886e4 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap complete.
I20240502 14:01:35.050251 15177 ts_tablet_manager.cc:1367] T 6aed24ef77e54c8b8ff09e6709a886e4 P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent bootstrapping tablet: real 0.025s user 0.022s sys 0.001s
I20240502 14:01:35.050784 15177 raft_consensus.cc:348] T 6aed24ef77e54c8b8ff09e6709a886e4 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:35.050932 15177 raft_consensus.cc:729] T 6aed24ef77e54c8b8ff09e6709a886e4 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e01cf117ef5d446590c06dd1a6ab7ef7, State: Initialized, Role: LEARNER
I20240502 14:01:35.051038 15177 consensus_queue.cc:260] T 6aed24ef77e54c8b8ff09e6709a886e4 P e01cf117ef5d446590c06dd1a6ab7ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:35.051323 15177 ts_tablet_manager.cc:1398] T 6aed24ef77e54c8b8ff09e6709a886e4 P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 14:01:35.052034 9454 tablet_copy_service.cc:342] P 1a490f170c614ee4900ef16c3287c7de: Request end of tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-6aed24ef77e54c8b8ff09e6709a886e4 received from {username='slave'} at 127.24.66.72:40361
I20240502 14:01:35.052138 9454 tablet_copy_service.cc:433] P 1a490f170c614ee4900ef16c3287c7de: ending tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-6aed24ef77e54c8b8ff09e6709a886e4 on tablet 6aed24ef77e54c8b8ff09e6709a886e4 with peer e01cf117ef5d446590c06dd1a6ab7ef7
I20240502 14:01:35.076112 9430 consensus_queue.cc:237] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 1.18, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } }
I20240502 14:01:35.078017 9566 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 18. Preceding OpId from leader: term: 1 index: 19. (index mismatch)
I20240502 14:01:35.078056 9165 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 18. Preceding OpId from leader: term: 1 index: 19. (index mismatch)
I20240502 14:01:35.078292 9037 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 18. Preceding OpId from leader: term: 1 index: 19. (index mismatch)
I20240502 14:01:35.078325 11360 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 18. Preceding OpId from leader: term: 1 index: 19. (index mismatch)
I20240502 14:01:35.078471 14297 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240502 14:01:35.078696 9697 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 18. Preceding OpId from leader: term: 1 index: 19. (index mismatch)
I20240502 14:01:35.078670 14922 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240502 14:01:35.078814 14174 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240502 14:01:35.079171 11100 raft_consensus.cc:1261] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 18. Preceding OpId from leader: term: 1 index: 19. (index mismatch)
I20240502 14:01:35.079668 14922 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240502 14:01:35.080657 14297 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240502 14:01:35.081519 14922 consensus_queue.cc:1035] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240502 14:01:35.082581 14297 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.19: config changed from index 18 to 19, VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:35.083804 9037 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.19: config changed from index 18 to 19, VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:35.084489 9169 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.19: config changed from index 18 to 19, VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:35.085750 9697 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.19: config changed from index 18 to 19, VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:35.085848 11360 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.19: config changed from index 18 to 19, VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:35.087245 8894 catalog_manager.cc:5484] T 8c10fb9748c747c6a89066fa113cda3c P bc9e4f2f668d48ea87d5aad4a2fb569d reported cstate change: config changed from index 18 to 19, VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) evicted. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:35.089362 9566 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Committing config change with OpId 1.19: config changed from index 18 to 19, VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:35.091725 15177 ts_tablet_manager.cc:902] T 655c86a7f52a40c68c3e95cb12286c29 P e01cf117ef5d446590c06dd1a6ab7ef7: Initiating tablet copy from peer 0488b0e4ed994db19d40aed8c15f945b (127.24.66.71:33679)
I20240502 14:01:35.092489 11211 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c10fb9748c747c6a89066fa113cda3c with delete_type TABLET_DATA_TOMBSTONED (TS e01cf117ef5d446590c06dd1a6ab7ef7 not found in new config with opid_index 19) from {username='slave'} at 127.0.0.1:46242
I20240502 14:01:35.092550 15177 tablet_copy_client.cc:308] T 655c86a7f52a40c68c3e95cb12286c29 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.71:33679
I20240502 14:01:35.092588 8882 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 8c10fb9748c747c6a89066fa113cda3c with cas_config_opid_index 18: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:35.093057 15202 tablet_replica.cc:330] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7: stopping tablet replica
I20240502 14:01:35.093207 15202 raft_consensus.cc:2229] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:35.093109 11100 raft_consensus.cc:2932] T 8c10fb9748c747c6a89066fa113cda3c P 304274b88d234198a3892e38c05014a5 [term 1 FOLLOWER]: Committing config change with OpId 1.19: config changed from index 18 to 19, VOTER e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "304274b88d234198a3892e38c05014a5" member_type: VOTER last_known_addr { host: "127.24.66.73" port: 42559 } attrs { promote: false } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } attrs { promote: false } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } }
I20240502 14:01:35.093727 9850 tablet_copy_service.cc:140] P 0488b0e4ed994db19d40aed8c15f945b: Received BeginTabletCopySession request for tablet 655c86a7f52a40c68c3e95cb12286c29 from peer e01cf117ef5d446590c06dd1a6ab7ef7 ({username='slave'} at 127.24.66.72:37729)
I20240502 14:01:35.093845 9850 tablet_copy_service.cc:161] P 0488b0e4ed994db19d40aed8c15f945b: Beginning new tablet copy session on tablet 655c86a7f52a40c68c3e95cb12286c29 from peer e01cf117ef5d446590c06dd1a6ab7ef7 at {username='slave'} at 127.24.66.72:37729: session id = e01cf117ef5d446590c06dd1a6ab7ef7-655c86a7f52a40c68c3e95cb12286c29
I20240502 14:01:35.094440 15202 raft_consensus.cc:2258] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:35.094949 9850 tablet_copy_source_session.cc:205] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:35.095176 15202 ts_tablet_manager.cc:1875] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:35.097070 15177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 655c86a7f52a40c68c3e95cb12286c29. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:35.099560 15177 tablet_copy_client.cc:783] T 655c86a7f52a40c68c3e95cb12286c29 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:35.099730 15177 tablet_copy_client.cc:647] T 655c86a7f52a40c68c3e95cb12286c29 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:35.100539 15202 ts_tablet_manager.cc:1888] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.18
I20240502 14:01:35.100623 15202 log.cc:1196] T 8c10fb9748c747c6a89066fa113cda3c P e01cf117ef5d446590c06dd1a6ab7ef7: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-7/wal/wals/8c10fb9748c747c6a89066fa113cda3c
I20240502 14:01:35.101169 8883 catalog_manager.cc:4830] TS e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72:35005): tablet 8c10fb9748c747c6a89066fa113cda3c (table rebalance_test_table_5 [id=ee30fa8091e1477f89045561437d7efa]) successfully deleted
I20240502 14:01:35.107159 15177 tablet_copy_client.cc:523] T 655c86a7f52a40c68c3e95cb12286c29 P e01cf117ef5d446590c06dd1a6ab7ef7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:35.108769 15177 tablet_bootstrap.cc:492] T 655c86a7f52a40c68c3e95cb12286c29 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap starting.
I20240502 14:01:35.158288 15177 tablet_bootstrap.cc:492] T 655c86a7f52a40c68c3e95cb12286c29 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:35.158676 15210 ts_tablet_manager.cc:902] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de: Initiating tablet copy from peer ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:35.158867 15177 tablet_bootstrap.cc:492] T 655c86a7f52a40c68c3e95cb12286c29 P e01cf117ef5d446590c06dd1a6ab7ef7: Bootstrap complete.
I20240502 14:01:35.159147 15177 ts_tablet_manager.cc:1367] T 655c86a7f52a40c68c3e95cb12286c29 P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent bootstrapping tablet: real 0.050s user 0.046s sys 0.004s
I20240502 14:01:35.159458 15210 tablet_copy_client.cc:308] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.67:34749
I20240502 14:01:35.159896 9322 tablet_copy_service.cc:140] P ccfb5bbc1bb54793812d11170ac2d25f: Received BeginTabletCopySession request for tablet 50451fdbd43b491fa701daac5de5e1c8 from peer 1a490f170c614ee4900ef16c3287c7de ({username='slave'} at 127.24.66.68:45369)
I20240502 14:01:35.159842 15177 raft_consensus.cc:348] T 655c86a7f52a40c68c3e95cb12286c29 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:35.160012 9322 tablet_copy_service.cc:161] P ccfb5bbc1bb54793812d11170ac2d25f: Beginning new tablet copy session on tablet 50451fdbd43b491fa701daac5de5e1c8 from peer 1a490f170c614ee4900ef16c3287c7de at {username='slave'} at 127.24.66.68:45369: session id = 1a490f170c614ee4900ef16c3287c7de-50451fdbd43b491fa701daac5de5e1c8
I20240502 14:01:35.160091 15177 raft_consensus.cc:729] T 655c86a7f52a40c68c3e95cb12286c29 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e01cf117ef5d446590c06dd1a6ab7ef7, State: Initialized, Role: LEARNER
I20240502 14:01:35.160249 15177 consensus_queue.cc:260] T 655c86a7f52a40c68c3e95cb12286c29 P e01cf117ef5d446590c06dd1a6ab7ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: NON_VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: true } }
I20240502 14:01:35.160995 9322 tablet_copy_source_session.cc:205] T 50451fdbd43b491fa701daac5de5e1c8 P ccfb5bbc1bb54793812d11170ac2d25f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:35.161523 15177 ts_tablet_manager.cc:1398] T 655c86a7f52a40c68c3e95cb12286c29 P e01cf117ef5d446590c06dd1a6ab7ef7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 14:01:35.162278 15210 ts_tablet_manager.cc:1875] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240502 14:01:35.162581 9850 tablet_copy_service.cc:342] P 0488b0e4ed994db19d40aed8c15f945b: Request end of tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-655c86a7f52a40c68c3e95cb12286c29 received from {username='slave'} at 127.24.66.72:37729
I20240502 14:01:35.162660 9850 tablet_copy_service.cc:433] P 0488b0e4ed994db19d40aed8c15f945b: ending tablet copy session e01cf117ef5d446590c06dd1a6ab7ef7-655c86a7f52a40c68c3e95cb12286c29 on tablet 655c86a7f52a40c68c3e95cb12286c29 with peer e01cf117ef5d446590c06dd1a6ab7ef7
I20240502 14:01:35.166345 15210 ts_tablet_manager.cc:1888] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240502 14:01:35.166532 15210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50451fdbd43b491fa701daac5de5e1c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:35.167441 15210 tablet_copy_client.cc:783] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:35.167585 15210 tablet_copy_client.cc:647] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:35.168818 15210 tablet_copy_client.cc:523] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:35.169807 15210 tablet_bootstrap.cc:492] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:35.190070 15210 tablet_bootstrap.cc:492] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:35.190392 15210 tablet_bootstrap.cc:492] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap complete.
I20240502 14:01:35.190591 15210 ts_tablet_manager.cc:1367] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.021s user 0.019s sys 0.000s
I20240502 14:01:35.191030 15210 raft_consensus.cc:348] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } }
I20240502 14:01:35.191190 15210 raft_consensus.cc:729] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: LEARNER
I20240502 14:01:35.191313 15210 consensus_queue.cc:260] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } }
I20240502 14:01:35.191622 15210 ts_tablet_manager.cc:1398] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 14:01:35.192245 9322 tablet_copy_service.cc:342] P ccfb5bbc1bb54793812d11170ac2d25f: Request end of tablet copy session 1a490f170c614ee4900ef16c3287c7de-50451fdbd43b491fa701daac5de5e1c8 received from {username='slave'} at 127.24.66.68:45369
I20240502 14:01:35.192400 9322 tablet_copy_service.cc:433] P ccfb5bbc1bb54793812d11170ac2d25f: ending tablet copy session 1a490f170c614ee4900ef16c3287c7de-50451fdbd43b491fa701daac5de5e1c8 on tablet 50451fdbd43b491fa701daac5de5e1c8 with peer 1a490f170c614ee4900ef16c3287c7de
I20240502 14:01:35.293781 15210 ts_tablet_manager.cc:902] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de: Initiating tablet copy from peer ccfb5bbc1bb54793812d11170ac2d25f (127.24.66.67:34749)
I20240502 14:01:35.294378 15210 tablet_copy_client.cc:308] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de: tablet copy: Beginning tablet copy session from remote peer at address 127.24.66.67:34749
I20240502 14:01:35.294927 9322 tablet_copy_service.cc:140] P ccfb5bbc1bb54793812d11170ac2d25f: Received BeginTabletCopySession request for tablet bfa2e6af0fd544fa8bfdaefedabb5e68 from peer 1a490f170c614ee4900ef16c3287c7de ({username='slave'} at 127.24.66.68:45369)
I20240502 14:01:35.295063 9322 tablet_copy_service.cc:161] P ccfb5bbc1bb54793812d11170ac2d25f: Beginning new tablet copy session on tablet bfa2e6af0fd544fa8bfdaefedabb5e68 from peer 1a490f170c614ee4900ef16c3287c7de at {username='slave'} at 127.24.66.68:45369: session id = 1a490f170c614ee4900ef16c3287c7de-bfa2e6af0fd544fa8bfdaefedabb5e68
I20240502 14:01:35.296487 9322 tablet_copy_source_session.cc:205] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 14:01:35.297457 15210 ts_tablet_manager.cc:1875] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240502 14:01:35.300601 15210 ts_tablet_manager.cc:1888] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.6
I20240502 14:01:35.300813 15210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfa2e6af0fd544fa8bfdaefedabb5e68. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 14:01:35.303053 15210 tablet_copy_client.cc:783] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de: tablet copy: Starting download of 0 data blocks...
I20240502 14:01:35.303315 15210 tablet_copy_client.cc:647] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de: tablet copy: Starting download of 1 WAL segments...
I20240502 14:01:35.304795 15210 tablet_copy_client.cc:523] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 14:01:35.305923 15210 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap starting.
I20240502 14:01:35.358444 15210 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 14:01:35.358850 15210 tablet_bootstrap.cc:492] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de: Bootstrap complete.
I20240502 14:01:35.359115 15210 ts_tablet_manager.cc:1367] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de: Time spent bootstrapping tablet: real 0.053s user 0.049s sys 0.000s
I20240502 14:01:35.359782 15210 raft_consensus.cc:348] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } }
I20240502 14:01:35.360006 15210 raft_consensus.cc:729] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1a490f170c614ee4900ef16c3287c7de, State: Initialized, Role: LEARNER
I20240502 14:01:35.361594 15210 consensus_queue.cc:260] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 1.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: NON_VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: true } }
I20240502 14:01:35.362983 15210 ts_tablet_manager.cc:1398] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240502 14:01:35.363750 9322 tablet_copy_service.cc:342] P ccfb5bbc1bb54793812d11170ac2d25f: Request end of tablet copy session 1a490f170c614ee4900ef16c3287c7de-bfa2e6af0fd544fa8bfdaefedabb5e68 received from {username='slave'} at 127.24.66.68:45369
I20240502 14:01:35.363855 9322 tablet_copy_service.cc:433] P ccfb5bbc1bb54793812d11170ac2d25f: ending tablet copy session 1a490f170c614ee4900ef16c3287c7de-bfa2e6af0fd544fa8bfdaefedabb5e68 on tablet bfa2e6af0fd544fa8bfdaefedabb5e68 with peer 1a490f170c614ee4900ef16c3287c7de
I20240502 14:01:35.476969 11229 raft_consensus.cc:1203] T 6aed24ef77e54c8b8ff09e6709a886e4 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 14:01:35.525162 11229 raft_consensus.cc:1203] T 655c86a7f52a40c68c3e95cb12286c29 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.10->[1.11-1.11] Dedup: 1.11->[]
I20240502 14:01:35.767292 9430 raft_consensus.cc:1203] T 50451fdbd43b491fa701daac5de5e1c8 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEARNER]: Deduplicated request from leader. Original: 1.7->[1.8-1.8] Dedup: 1.8->[]
I20240502 14:01:35.843003 9430 raft_consensus.cc:1203] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEARNER]: Deduplicated request from leader. Original: 1.13->[1.14-1.14] Dedup: 1.14->[]
I20240502 14:01:35.985487 14922 raft_consensus.cc:1050] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de: attempting to promote NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 to VOTER
I20240502 14:01:35.985962 14922 consensus_queue.cc:237] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } }
I20240502 14:01:35.987811 11360 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:35.988068 9035 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:35.988232 9829 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:35.988487 9165 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:35.988476 15320 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:35.988708 11229 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:35.988875 15320 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:35.989189 15320 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:35.989498 15320 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:35.989640 9694 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:35.990190 9300 raft_consensus.cc:1261] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 1a490f170c614ee4900ef16c3287c7de: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20240502 14:01:35.990864 15319 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:35.991039 15321 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:35.992986 14297 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:35.994868 11229 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:35.995671 9165 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:35.996986 9035 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:35.997264 8890 catalog_manager.cc:5484] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de reported cstate change: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1a490f170c614ee4900ef16c3287c7de" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20240502 14:01:35.998510 9829 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:36.000420 15318 consensus_queue.cc:1035] T 6aed24ef77e54c8b8ff09e6709a886e4 P 1a490f170c614ee4900ef16c3287c7de [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 14:01:36.005442 11360 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:36.005887 9694 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:36.005890 9300 raft_consensus.cc:2932] T 6aed24ef77e54c8b8ff09e6709a886e4 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 8 to 9, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:36.111145 14956 raft_consensus.cc:1050] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b: attempting to promote NON_VOTER e01cf117ef5d446590c06dd1a6ab7ef7 to VOTER
I20240502 14:01:36.111680 14956 consensus_queue.cc:237] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } }
I20240502 14:01:36.114161 9300 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:36.114262 9430 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:36.114377 9165 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:36.114562 11229 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 LEARNER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:36.114753 15338 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:36.115056 15337 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:36.115128 9035 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:36.115234 15336 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:36.115376 15338 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:36.115500 11360 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:36.115526 9697 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20240502 14:01:36.115929 15336 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:36.116243 15336 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:36.118361 15338 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 14:01:36.118857 15337 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 11 to 12, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:36.120002 9165 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:36.120740 11360 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:36.121328 11229 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:36.122107 9430 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:36.122184 9035 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:36.124120 8894 catalog_manager.cc:5484] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: config changed from index 11 to 12, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:36.126417 9697 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:36.126428 9300 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 11 to 12, e01cf117ef5d446590c06dd1a6ab7ef7 (127.24.66.72) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false replace: true } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:36.134907 9829 consensus_queue.cc:237] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } }
I20240502 14:01:36.136199 11229 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:36.136495 9697 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:36.137063 9431 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:36.137146 9035 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:36.137495 15338 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:36.137696 15339 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:36.137909 14955 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:36.138028 15336 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:36.138455 9165 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:36.138795 14955 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:36.140389 9300 raft_consensus.cc:1261] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Refusing update from remote peer 0488b0e4ed994db19d40aed8c15f945b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 1 index: 13. (index mismatch)
I20240502 14:01:36.140830 15337 consensus_queue.cc:1035] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 14:01:36.141412 15338 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P 0488b0e4ed994db19d40aed8c15f945b [term 1 LEADER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:36.141999 9430 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P 1a490f170c614ee4900ef16c3287c7de [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:36.141964 9165 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:36.144196 9300 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P ccfb5bbc1bb54793812d11170ac2d25f [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:36.145153 11229 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:36.145452 8891 catalog_manager.cc:5484] T 655c86a7f52a40c68c3e95cb12286c29 P d83af1bc823a4f2ca531f0d6825f8e16 reported cstate change: config changed from index 12 to 13, VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) evicted. New cstate: current_term: 1 leader_uuid: "0488b0e4ed994db19d40aed8c15f945b" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:36.147217 9697 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:36.148684 11342 tablet_service.cc:1514] Processing DeleteTablet for tablet 655c86a7f52a40c68c3e95cb12286c29 with delete_type TABLET_DATA_TOMBSTONED (TS 535b19a469c34efaaf49ca1cc2e1e5fc not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:51718
I20240502 14:01:36.148869 8880 catalog_manager.cc:4997] ChangeConfig:REMOVE_PEER RPC for tablet 655c86a7f52a40c68c3e95cb12286c29 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20240502 14:01:36.149341 15357 tablet_replica.cc:330] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc: stopping tablet replica
I20240502 14:01:36.149513 15357 raft_consensus.cc:2229] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 14:01:36.149986 15357 raft_consensus.cc:2258] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 14:01:36.150401 9035 raft_consensus.cc:2932] T 655c86a7f52a40c68c3e95cb12286c29 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Committing config change with OpId 1.13: config changed from index 12 to 13, VOTER 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "d83af1bc823a4f2ca531f0d6825f8e16" member_type: VOTER last_known_addr { host: "127.24.66.66" port: 38837 } } peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } }
I20240502 14:01:36.150889 15357 ts_tablet_manager.cc:1875] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 14:01:36.153488 15357 ts_tablet_manager.cc:1888] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.12
I20240502 14:01:36.153578 15357 log.cc:1196] T 655c86a7f52a40c68c3e95cb12286c29 P 535b19a469c34efaaf49ca1cc2e1e5fc: Deleting WAL directory at /tmp/dist-test-taskfX8GOh/test-tmp/rebalancer_tool-test.7.LocationAwareRebalancingParamTest.RebalanceWithIgnoredTServer_idx6_rf7_t7_l_3_4_3.1714658379925612-24841-0/raft_consensus-itest-cluster/ts-9/wal/wals/655c86a7f52a40c68c3e95cb12286c29
I20240502 14:01:36.154208 8883 catalog_manager.cc:4830] TS 535b19a469c34efaaf49ca1cc2e1e5fc (127.24.66.74:34421): tablet 655c86a7f52a40c68c3e95cb12286c29 (table rebalance_test_table_3 [id=e3277045589d46628bee568b257ed243]) successfully deleted
I20240502 14:01:36.311084 14813 raft_consensus.cc:1050] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f: attempting to promote NON_VOTER 1a490f170c614ee4900ef16c3287c7de to VOTER
I20240502 14:01:36.311602 14813 consensus_queue.cc:237] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 1.14, Last appended by leader: 0, Current term: 1, Majority size: 5, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 } } peers { permanent_uuid: "ccfb5bbc1bb54793812d11170ac2d25f" member_type: VOTER last_known_addr { host: "127.24.66.67" port: 34749 } } peers { permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 } } peers { permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true } } peers { permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 } } peers { permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72" port: 35005 } attrs { promote: false } } peers { permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false } } peers { permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false } }
I20240502 14:01:36.313580 9829 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 0488b0e4ed994db19d40aed8c15f945b [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 14:01:36.313686 9566 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P d0f45862d7f14ce1976c1a37842f25ca [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 14:01:36.313984 9430 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 1a490f170c614ee4900ef16c3287c7de [term 1 LEARNER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 14:01:36.314045 14797 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f45862d7f14ce1976c1a37842f25ca" member_type: VOTER last_known_addr { host: "127.24.66.69" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 14:01:36.314294 15375 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0488b0e4ed994db19d40aed8c15f945b" member_type: VOTER last_known_addr { host: "127.24.66.71" port: 33679 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 14:01:36.314527 14797 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a490f170c614ee4900ef16c3287c7de" member_type: VOTER last_known_addr { host: "127.24.66.68" port: 42907 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 14:01:36.316190 9035 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P bc9e4f2f668d48ea87d5aad4a2fb569d [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 14:01:36.316637 14797 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9e4f2f668d48ea87d5aad4a2fb569d" member_type: VOTER last_known_addr { host: "127.24.66.65" port: 45459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 14:01:36.317140 9697 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P efa836edab8d4853a7ca47a23117e288 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 14:01:36.317159 11360 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P 535b19a469c34efaaf49ca1cc2e1e5fc [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 14:01:36.317577 15375 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa836edab8d4853a7ca47a23117e288" member_type: VOTER last_known_addr { host: "127.24.66.70" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 14:01:36.317763 15374 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "535b19a469c34efaaf49ca1cc2e1e5fc" member_type: VOTER last_known_addr { host: "127.24.66.74" port: 34421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 14:01:36.317904 11229 raft_consensus.cc:1261] T bfa2e6af0fd544fa8bfdaefedabb5e68 P e01cf117ef5d446590c06dd1a6ab7ef7 [term 1 FOLLOWER]: Refusing update from remote peer ccfb5bbc1bb54793812d11170ac2d25f: Log matching property violated. Preceding OpId in replica: term: 1 index: 14. Preceding OpId from leader: term: 1 index: 15. (index mismatch)
I20240502 14:01:36.319257 15375 consensus_queue.cc:1035] T bfa2e6af0fd544fa8bfdaefedabb5e68 P ccfb5bbc1bb54793812d11170ac2d25f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e01cf117ef5d446590c06dd1a6ab7ef7" member_type: VOTER last_known_addr { host: "127.24.66.72"